diff --git a/configs/AM62LX/AM62LX_linux_toc.txt b/configs/AM62LX/AM62LX_linux_toc.txt index 805510b85..f698c8d9c 100644 --- a/configs/AM62LX/AM62LX_linux_toc.txt +++ b/configs/AM62LX/AM62LX_linux_toc.txt @@ -141,6 +141,7 @@ common/How_to_Guides/AM62Lx_EVM_Hardware_Setup linux/Demo_User_Guides/index_Demos linux/Demo_User_Guides/TI_LVGL_Demo_User_Guide linux/Demo_User_Guides/EV_charging_Demo_User_Guide +linux/Demo_User_Guides/Webserver_Demo_User_Guide boards/index boards/beagle/BeagleBadge/BeagleBadge diff --git a/configs/AM62PX/AM62PX_linux_toc.txt b/configs/AM62PX/AM62PX_linux_toc.txt index 9a247a5b7..ea88d28b6 100644 --- a/configs/AM62PX/AM62PX_linux_toc.txt +++ b/configs/AM62PX/AM62PX_linux_toc.txt @@ -188,5 +188,6 @@ linux/Demo_User_Guides/Seva_Store linux/Demo_User_Guides/Display_Cluster_User_Guide linux/Demo_User_Guides/Chromium_Browser linux/Demo_User_Guides/GUI_Frameworks_User_Guide +linux/Demo_User_Guides/Webserver_Demo_User_Guide linux/Documentation_Tarball diff --git a/configs/AM62X/AM62X_linux_toc.txt b/configs/AM62X/AM62X_linux_toc.txt index ab66e95c6..07b01f227 100644 --- a/configs/AM62X/AM62X_linux_toc.txt +++ b/configs/AM62X/AM62X_linux_toc.txt @@ -191,6 +191,7 @@ linux/Demo_User_Guides/TI_Apps_Launcher_User_Guide linux/Demo_User_Guides/Seva_Store linux/Demo_User_Guides/Chromium_Browser linux/Demo_User_Guides/GUI_Frameworks_User_Guide +linux/Demo_User_Guides/Webserver_Demo_User_Guide common/PRU-ICSS/Getting_Started_with_PSSP common/PRU-ICSS/Header_Files diff --git a/source/images/Webserver_CPU_Performance.PNG b/source/images/Webserver_CPU_Performance.PNG deleted file mode 100644 index 08f331648..000000000 Binary files a/source/images/Webserver_CPU_Performance.PNG and /dev/null differ diff --git a/source/images/Webserver_CPU_Performance.png b/source/images/Webserver_CPU_Performance.png new file mode 100644 index 000000000..4bf03f411 Binary files /dev/null and b/source/images/Webserver_CPU_Performance.png differ diff --git a/source/images/Webserver_audio_classification.PNG b/source/images/Webserver_audio_classification.PNG deleted file mode 100644 index 7ea5c5b80..000000000 Binary files a/source/images/Webserver_audio_classification.PNG and /dev/null differ diff --git a/source/images/Webserver_audio_classification.png b/source/images/Webserver_audio_classification.png new file mode 100644 index 000000000..c6ff9d90c Binary files /dev/null and b/source/images/Webserver_audio_classification.png differ diff --git a/source/images/Webserver_home_page.PNG b/source/images/Webserver_home_page.PNG deleted file mode 100644 index 934d3ebde..000000000 Binary files a/source/images/Webserver_home_page.PNG and /dev/null differ diff --git a/source/images/Webserver_home_page.png b/source/images/Webserver_home_page.png new file mode 100644 index 000000000..1ef847094 Binary files /dev/null and b/source/images/Webserver_home_page.png differ diff --git a/source/images/Webserver_speech_to_text.png b/source/images/Webserver_speech_to_text.png new file mode 100644 index 000000000..20dd72210 Binary files /dev/null and b/source/images/Webserver_speech_to_text.png differ diff --git a/source/linux/Demo_User_Guides/Webserver_Demo_User_Guide.rst b/source/linux/Demo_User_Guides/Webserver_Demo_User_Guide.rst index aa242e371..d31c7b890 100644 --- a/source/linux/Demo_User_Guides/Webserver_Demo_User_Guide.rst +++ b/source/linux/Demo_User_Guides/Webserver_Demo_User_Guide.rst @@ -8,96 +8,151 @@ Web server demo - User Guide Overview ******** -This document describes the Out of Box Web Server Demo Application, which the Linux -SDK delivers with the |__SDK_FULL_NAME__| |__PART_FAMILY_NAME__|, applicable to boards -such as the `BEAGL-BONE-GRN-ECO `__ and -`TMDXEVM3358 `__. The main purpose of this demo is to -showcase a glimpse of the ARM analytics AI stack integrated into the filesystem, -providing an out-of-box experience even users do not connect a display to the board. +The Out-of-Box (OOB) Web Server Demo provides a browser-accessible interface +for exploring key capabilities of the |__PART_FAMILY_NAME__| platform. A +lightweight HTTP server runs on the board and serves a dynamic web application. +Users can interact with the board from any PC on the same network without +requiring a connected display. -The demo showcases a web server running on an AM335x based board, providing a web interface with three key sections: +The demo uses a plugin architecture that adapts automatically to each target +device. The following table lists the demos available on |__PART_FAMILY_NAME__|: -* **Audio Classification Demo**: Users can connect a USB audio capture device to run real-time audio classification, displaying classification statistics. -* **Live CPU Performance Metrics**: Provides a real-time CPU usage indicator, historical usage data for the last 5 minutes, and detailed CPU information. -* **Documentation Links**: Offers convenient access to relevant documentation. +.. list-table:: Available Demos by Platform + :widths: 40 10 10 10 10 + :header-rows: 1 -This document provides all necessary equipment requirements and instructions. + * - Demo + - AM335x + - AM62x + - AM62Px + - AM62Lx + * - :ref:`webserver-cpu-monitor` + - Yes + - Yes + - Yes + - Yes + * - :ref:`webserver-audio-classification` + - Yes + - Yes + - Yes + - Yes + * - :ref:`webserver-speech-to-text` + - No + - Yes + - Yes + - Yes + +.. note:: + + This guide uses screenshots from AM62Px. The web interface layout is the + same across all supported platforms; the device name and board image shown + on the home page will reflect the actual device in use. ********************** Hardware Prerequisites ********************** -- TI AM335x based board (e.g., `BEAGL-BONE-GRN-ECO `__ and `TMDXEVM3358 `__) - (Note: For the Beaglebone Green Eco, a display is available via an `HDMI cape `__) +- |__PART_FAMILY_NAME__| evaluation board - PC (Windows or Linux) -- Ethernet cables -- Ethernet switch or ethernet router with DHCP service -- SD card (minimum 16GB) -- Audio Capture Device +- Ethernet cable +- Ethernet switch or router with DHCP service +- SD card (minimum 32 GB) +- Audio capture device (required for the Audio Classification and Speech to Text demos) *********** Get Started *********** -#. Flash an SD card with the :file:`tisdk-default-image`. User can download the :file:`tisdk-default-image` wic - image from |__SDK_DOWNLOAD_URL__|. Please follow the instructions from here to :ref:`Flash an SD card `. +#. Flash an SD card with the :file:`tisdk-default-image`. Download the + :file:`tisdk-default-image` wic image from |__SDK_DOWNLOAD_URL__|. Follow + the instructions at :ref:`Flash an SD card `. + +#. Insert the SD card into the |__PART_FAMILY_NAME__| board and set it to boot + from SD card. + +#. Connect an Ethernet cable from your switch or router to the board. + +#. Connect your PC to the same Ethernet switch or router. + +#. Connect the UART to the PC USB port. -#. Insert the SD card into the AM335x based board and set it to boot via SD card Boot mode. +#. Open a terminal program (such as TeraTerm or minicom) and connect to the + serial port at 115200 bps, 8 data bits, no parity, 1 stop bit, no flow + control. -#. Connect an ethernet cable from your ethernet switch or router to the - AM335x based board. +#. Power on the board. -#. Connect your PC to the same ethernet switch or router. +#. After Linux boot completes, log in as ``root``. Use the + :command:`ifconfig` command to find the board's IP address. -#. Connect the UART to the PC's USB port. +#. On the host PC, open a browser and navigate to: + ``http://:3000`` -#. Open a terminal program (like TeraTerm or minicom) and connect to the - serial port. Set the port to 115200bps, 8 bit, no parity, 1 stop bit, no flow control. +#. The home page displays, showing the demos available for your device. -#. Power on the AM335x based board. + .. image:: /images/Webserver_home_page.png + :alt: Webserver Demo Home Page + :width: 75% -#. After the Linux boot completes, login as "root". Use the :command:`ifconfig` command - to find out the IP address of the device. +.. _webserver-cpu-monitor: + +**************************** +Live CPU Performance Metrics +**************************** -#. On the host PC, open a Internet Browser and enter in the following: ``http://:3000`` +Available on all supported platforms. -#. The following web page will be displayed, the home page shows as below. +The CPU monitor provides a real-time view of processor usage. -.. image:: /images/Webserver_home_page.PNG - :alt: Webserver Demo Page +.. image:: /images/Webserver_CPU_Performance.png + :alt: CPU Performance Metrics :width: 75% +* **CPU Usage**: A real-time gauge showing the current CPU usage. +* **CPU History**: A graph displaying CPU usage over the last 5 minutes. +* **Statistics**: Average and maximum CPU usage calculated from the history data. + +.. _webserver-audio-classification: + ************************* Audio Classification Demo ************************* -The web interface provides an audio classification demo that uses the integrated ARM analytics AI stack. +Available on: AM335x, AM62x, AM62Px, AM62Lx. -.. image:: /images/Webserver_audio_classification.PNG +The audio classification demo uses the integrated AI stack to classify sounds +captured from a microphone in real time. + +.. image:: /images/Webserver_audio_classification.png :alt: Audio Classification Demo :width: 75% To use the demo: -#. Connect a USB audio device (e.g., a USB headset) to the AM335x based board. -#. Click the "Refresh Devices" button to detect the connected audio capture device. -#. Select the desired audio capture device from the list of available devices. -#. Click the "Start Classification" button to begin real-time audio classification. -#. The "Live Classification" section will display the current classification, total classifications, unique classes, session time, and updates per minute. -#. A "Classification History" log is also available, showing a timestamped record of classifications. -#. Click the "Stop Classification" button to end the demo. +#. Connect an audio capture device to the board. +#. Click **Refresh Devices** to detect the connected audio capture device. +#. Select the audio capture device from the list. +#. Click **Start Classification** to begin real-time audio classification. +#. The **Live Classification** section displays the current classification, + total classifications, unique classes, session time, and updates per minute. +#. The **Classification History** log shows a timestamped record of all + classifications. +#. Click **Stop Classification** to end the demo. **Technical Details** -The audio classification demo leverages :ref:`NNStreamer `, a GStreamer-based neural network framework, to create and run the processing pipeline. +The audio classification demo uses :ref:`NNStreamer `, a +GStreamer-based neural network framework, to build and run the inference +pipeline. -* **Model:** The demo uses the `YAMNet `__ sound classification model. -* **Deep Learning Runtime:** The model is executed using the :ref:`TensorFlow Lite ` runtime. +* **Model:** `YAMNet `__ + sound classification model + (`yamnet_audio_classification.tflite `__). +* **Deep Learning Runtime:** :ref:`TensorFlow Lite `. +* **Inference:** XNNPACK delegate on the Arm Cortex-A cores. **GStreamer Pipeline** -The following GStreamer pipeline is used to process the audio from the microphone, run the inference, and decode the results: - .. code-block:: console gst-launch-1.0 alsasrc device= ! \ @@ -108,59 +163,126 @@ The following GStreamer pipeline is used to process the audio from the microphon tensor_transform mode=arithmetic option=typecast:float32,add:0.5,div:32767.5 ! \ tensor_transform mode=transpose option=1:0:2:3 ! \ queue leaky=2 max-size-buffers=10 ! \ - tensor_filter framework=tensorflow2-lite model=/usr/share/oob-demo-assets/models/yamnet_audio_classification.tflite custom=Delegate:XNNPACK,NumThreads:2 ! \ - tensor_decoder mode=image_labeling option1=/usr/share/oob-demo-assets/labels/yamnet_label_list.txt ! \ + tensor_filter framework=tensorflow2-lite \ + model=/usr/share/oob-demo-assets/models/yamnet_audio_classification.tflite \ + custom=Delegate:XNNPACK,NumThreads:2 ! \ + tensor_decoder mode=image_labeling \ + option1=/usr/share/oob-demo-assets/labels/yamnet_label_list.txt ! \ filesink buffer-mode=2 location= -**************************** -Live CPU Performance Metrics -**************************** +.. _webserver-speech-to-text: -The web interface also provides a live view of the CPU performance metrics. +******************* +Speech to Text Demo +******************* -.. image:: /images/Webserver_CPU_Performance.PNG - :alt: CPU Performance Metrics +Available on: AM62x, AM62Px, AM62Lx. + +The speech-to-text demo uses an on-device speech recognition model to +transcribe spoken words captured from a microphone in real time. + +.. image:: /images/Webserver_speech_to_text.png + :alt: Speech to Text Demo :width: 75% -This section includes: +To use the demo: + +#. Connect an audio capture device to the board. +#. Click **Refresh Devices** to detect connected audio capture devices. +#. Select the microphone from the list. +#. Click **Start** to begin transcription. Transcribed text is displayed in + the output area as you speak. +#. Click **Stop** to end the demo. + +**Technical Details** + +The speech-to-text demo uses :ref:`NNStreamer ` and +:ref:`ONNX Runtime ` to process audio through a streaming +inference pipeline. + +* **Model:** `Silero STT en_v5 `__ + English speech recognition model + (`en_v5_static.onnx `__). +* **Deep Learning Runtime:** :ref:`ONNX Runtime `. +* **Decoder:** Greedy CTC decoder over a 999-token BPE vocabulary, with + blank-ratio silence gating to suppress output during microphone silence + or background noise. Implemented in :command:`speech_utils` using the + ``tensor_sink`` new-data signal. + +**GStreamer Pipeline** + +The pipeline processes audio in a 3-second sliding window with a 1-second +stride, enabling low-latency continuous transcription. +:command:`speech_utils` builds the pipeline internally through the GStreamer +API; the ``tensor_sink`` new-data callback performs CTC decoding and writes +transcripts to a named FIFO. + +.. code-block:: console + + gst-launch-1.0 alsasrc device= ! \ + audioconvert ! audioresample ! \ + audio/x-raw,format=S16LE,channels=1,rate=16000 ! \ + tensor_converter frames-per-tensor=16000 ! \ + tensor_aggregator frames-in=16000 frames-out=48000 \ + frames-flush=16000 frames-dim=1 ! \ + tensor_transform mode=arithmetic option=typecast:float32,div:32768 ! \ + tensor_filter framework=onnxruntime \ + model=/usr/share/oob-demo-assets/models/en_v5_static.onnx ! \ + fakesink -* **CPU Usage**: A real-time gauge showing the current CPU utilization. -* **CPU History**: A graph displaying the CPU usage history over the last 5 minutes. -* **Statistics**: Average and maximum CPU usage calculated for the data from CPU History data. ********************* Software Architecture ********************* -The demo consists of three main components: a web interface, a backend web server, and a set of Linux applications. +The demo consists of three main layers: a web front end, a backend HTTP +server, and device-specific Linux applications. -* **Web Interface (GUI):** A dynamic HTML page with JavaScript that runs in the user's browser. It provides the user interface for the audio classification demo and CPU performance metrics, making asynchronous requests to the web server for data updates. +* **Web front end:** A dynamic HTML page with JavaScript that runs in the + user's browser. It makes asynchronous REST and WebSocket requests to the + server to update the display in real time. -* **Web Server:** A lightweight HTTP server (e.g., Node.js) running on the AM335x's Arm A8 core. It serves the static web page and provides a simple REST API for the frontend to interact with the underlying Linux applications. +* **Web Server:** A Node.js Express server running on the board. It serves the + static web page and loads demo plugins at startup based on the device + configuration in :file:`/usr/share/webserver-oob/app/device.json`. Each + plugin registers its own REST and WebSocket endpoints. -* **Linux Applications:** A set of background applications running on the AM335x, with the following key functions: +* **Linux Applications:** Native C utilities built for the target device: - * A C application that reads real-time system information (like CPU stats from :file:`/proc/stat`) and makes it available to the web server. - * The audio classification demo, which is a GStreamer pipeline that leverages the NNStreamer framework. This pipeline reads from a USB audio device, processes the audio with the YAMNet model using the TensorFlow Lite runtime, and sends the classification results to the web server. + * :command:`cpu_stats` - reads CPU load from :file:`/proc/stat` and + exposes it to the web server. + * :command:`audio_utils` - manages the GStreamer+NNStreamer audio + classification pipeline and writes results to a named FIFO. + * :command:`speech_utils` - manages the GStreamer+ONNX Runtime + speech-to-text pipeline and writes transcripts to a named FIFO. ******************* Directory Structure ******************* -Yocto recipe for building this demo can be found at `github:webserver-oob_git.bb `__. +The Yocto recipe for building this demo is at +`github:webserver-oob_git.bb `__. -The source code for the demo can be found at `github/TexasInstruments/webserver-oob-demo `__. -For instructions on building the utilities see the `README `__ file. +The source code is at +`webserver-oob-demo on GitHub `__. +See the `README `__ +for build instructions. .. list-table:: Directory Structure - :widths: 20 30 + :widths: 30 70 :header-rows: 1 - * - Directory Name + * - Directory - Description - * - :file:`webserver_app/app` - - GUI code (HTML, CSS, JavaScript) - * - :file:`webserver_app/linux_app` - - Linux application code (C code) - * - :file:`webserver_app/webserver` - - Web server code (Node.js) + * - :file:`common/app` + - Generic front end (HTML, CSS, JavaScript) + * - :file:`common/linux_app` + - Shared C utilities: :command:`cpu_stats`, :command:`audio_utils`, + :command:`speech_utils` + * - :file:`common/webserver` + - Express server and demo plugin loader + * - :file:`demos/` + - Per-demo plugin: :file:`manifest.json` and :file:`server-plugin.js` + * - :file:`devices/` + - Per-device metadata (:file:`device.json`), front end overlay, and + device-specific Linux application build