1. Click File in the top-left corner, then select Preferences.
2. Look for the field labeled "Additional boards manager URLs" near the bottom of the window.
3. Copy and paste this exact link into that box:
   https://espressif.github.io/arduino-esp32/package_esp32_index.json
4. Click OK to close the Preferences window.
5. Now, on the left side of the screen, click the Boards Manager icon (it looks like a file folder with a little board inside, or go to Tools > Board > Boards Manager...).
6. In the search bar at the top of the Boards Manager, type: esp32
7. Look for the result simply titled "esp32 by Espressif Systems" and click Install. This download might take a few minutes!

Next, we need to install the specific code libraries that tell the board how to talk to the HUB75 LED matrices, the (optional) inner OLED screen, the built-in accelerometer, and the (optional) boop light sensor.

1. Click the Library Manager icon on the left side of the screen (it looks like a stack of books, or go to Sketch > Include Library > Manage Libraries...).
2. In the search bar, search for and install each of these five libraries:
   • Adafruit GFX Library (by Adafruit)
   • Adafruit SSD1306 (by Adafruit)
   • ESP32 HUB75 LED MATRIX PANEL DMA Display (by mrcodetastic)
   • Adafruit VEML7700 Library (by Adafruit)
   • Adafruit LIS3DH (by Adafruit)
3. Note: If Arduino pops up a window asking if you want to install "missing dependencies" (such as Adafruit BusIO or Adafruit Unified Sensor) while installing these, always click "Install All"! Even if you do not have an OLED screen or a VEML7700 boop light sensor plugged into your board, you still need these libraries installed so the firmware code can compile. On startup, the firmware automatically checks which sensors are connected and will happily run without any that aren't plugged in!

1. Go up to the top menu and click Tools > Board > esp32.
2. Scroll down that long list until you find "Adafruit MatrixPortal ESP32-S3" and click it.
3. Go back to Tools > Port. You should see a COM port listed there (it might even say "Adafruit MatrixPortal S3" next to it). Click it to select it. (If you don't see a port, your USB cable might be power-only, or the board isn't plugged in fully!)

1. Go to File > New Sketch to open a blank editor.
2. Delete any default code that is already sitting in the window so it is completely empty.
3. Copy and paste the custom firmware code I have written (ProtoStreamESP32FirmwareVBP.txt) directly into the main editor window.

   Note: By default, the firmware is set to display the LED matrices at maximum brightness (255). If you want to reduce the brightness, you can change the display->setBrightness8(255); value on line 168 to a lower number between 0 and 255. I recommend keeping it at the default value even if it appears extremely bright on your desk, as the semi-opaque outermost visor will dim it significantly.

4. Finally, click the right-facing arrow (Upload) in the very top-left corner of the IDE.

   Note: If you are receiving an upload error (such as PermissionError(13, 'A device attached to the system is not functioning.', None, 31)), you may need to enter ROM bootloader mode. This can be achieved by pressing and holding the oval-shaped BOOT button near the middle-left of the board, then—while keeping BOOT held—pressing and releasing the button labeled RESET near the top-left of the board one time, and finally releasing the BOOT button. You will know you have entered ROM bootloader mode when the status NeoPixel LED displays a solid color (rather than flashing as it was before). After entering ROM bootloader mode, click Tools > Port, select the new port that has appeared in the list (it usually says something like FamilyController), and click Upload again.

The Arduino IDE will now compile the code (which might take a minute or two) and write it to your board. Look at the black output box at the bottom of the screen. If it says "Done uploading," you are good to go!

When you launch ProtoStream for the very first time, the app will make sure you have all the permissions needed for its features to work properly:

• Microphone (used by Audio Visualizer and Vocal Sync)
• Location & Nearby Devices (required by Android to scan for and bind directly to your ESP32's local Wi-Fi hotspot)
• Photos and Videos (used to import your custom faces, emotes, and GIFs)

Simply tap each permission button in the pop-up panel; as you grant each one, its button will turn green, and the panel will automatically close once all permissions are granted.

Right after that, I (well, a digital version of me—Yen!) will pop up on screen to offer you an interactive, step-by-step tour of the entire ProtoStream app! The tour walks you through choosing your language, connecting to your visor, using the radial menus, and testing out every major feature hands-on.

Special thanks to gigasrock for creating the assets used to guide you around.

• Want to skip the tour? If you're already a pro (or just want me to stop yapping), simply tap directly on Yen's face when prompted and confirm that you'd like to exit!
• Need a refresher later? You can replay the entire tour—or jump straight into a mini-tutorial for one specific feature—at any time by opening Settings and tapping the "Help & Tour Guide" button at the top of the page.

Once your ESP32 is flashed with the firmware and your phone has ProtoStream installed, you are ready to pair the two!

1. Power on your ESP32 board by plugging a USB-C cable into it connected to your computer or to a 5V portable power bank (like a standard phone charger). It will automatically display its startup face and create a Wi-Fi hotspot named "MatrixStreamer" (with the default password "ProtoStream").
2. Open the ProtoStream app and look at the top navigation bar. Here you will see three buttons:
   • Connect to Wi-Fi (the large blue button on the left)
   • D/C (the red-outlined disconnect button in the middle)
   • Settings (the gear icon button on the right)
3. Tap the "Connect to Wi-Fi" button. A banner at the top will say "Searching for MatrixStreamer", and after a few seconds, your phone will automatically connect and bind to the ESP32's hotspot!
4. You will know you are connected when a toast message appears saying "Connected & Bound to MatrixStreamer", and the Connection Status Indicator right underneath the top buttons changes from a red dot ("Disconnected from Protogen") to a glowing green dot ("Connected to Protogen").
5. When you are done using your visor and want to sever the connection, simply tap the "D/C" button at the top of the screen.

A Quick Connection Tip: You may occasionally have to tap "Connect to Wi-Fi" twice if your phone was already actively connected to your home Wi-Fi network prior to tapping the button. Using the in-app "Connect to Wi-Fi" button is strongly recommended because it binds the app's data socket directly to the visor while still allowing your phone to use mobile data! If your phone model refuses to connect through the in-app prompt, you can manually connect to the "MatrixStreamer" network in your phone's system Wi-Fi settings instead (though doing so manually may temporarily pause your internet access while connected to the visor).

Note on Customizing Your Wi-Fi Name & Password: If you plan on wearing your suit at a convention where dozens of other Protogens might also be using ProtoStream, you will definitely want to customize your Wi-Fi network name and password so nobody else accidentally connects to your face!

• To do this, open the firmware code in the Arduino IDE and edit the ap_ssid ("MatrixStreamer") and ap_pass ("ProtoStream") variables on lines 44 and 45 before uploading it to your ESP32 (your password must be at least 8 characters long).
• Then, open the ProtoStream app on your phone, tap the Settings gear icon in the top-right corner, scroll to the "ESP32 Wi-Fi Setup" card, type your new SSID and Password into the boxes, and tap "Save".
• Already flashed your firmware, but want to change the network name and password later? No worries—just edit the code and flash it again! Any subsequent flash cleanly overwrites the previous one.

Instead of cluttering your screen with dozens of tiny buttons, ProtoStream organizes its tools into three circular radial menu buttons stacked vertically in the middle of the main screen:

1. Main Radial (Top circular button with the ProtoStream app icon)
2. Misc. Radial (Middle circular button)
3. Display Mode (Bottom circular button displaying your active Display Mode icon)

To use any radial menu:

1. Tap and HOLD your finger on the circular button. A wheel of icons will smoothly pop out around your finger.
2. Without lifting your finger, SWIPE in the direction of the feature you want to use.
3. RELEASE your finger to launch that feature! (If you change your mind, just slide your finger back to the center or away from the icons before letting go.)

Here is where everything lives by default:

• Main Radial Menu (Your core streaming and voice/audio tools):
  • Swipe UP (12 o'clock) — Start Live Screen Streaming: Mirrors your phone's entire screen (or a chosen app) live onto your visor in real time.
  • Swipe DOWN (6 o'clock) — Stop Streaming / Clear: Immediately stops your active screen stream or static image stream.
  • Swipe LEFT (9 o'clock) — Audio Visualizer: Opens the Visualizer Settings pop-up to turn your visor into a sound-reactive equalizer or beat-synced GIF display.
  • Swipe RIGHT (3 o'clock) — Vocal Sync: Opens the voice-activated expression switcher.
• Misc. Radial Menu (Your creative tools, emote switcher, and games):
  • Swipe UP (12 o'clock) — Emote Panel: Opens the instant emote switcher with controller keybind support.
  • Swipe DOWN-LEFT (8 o'clock) — Drawing: Launches the built-in pixel art studio.
  • Swipe DOWN-RIGHT (4 o'clock) — Games Plugin Manager: Opens the visor game arcade and custom game loader.
• Display Mode Radial Menu (Quick display layout switcher):
  • Swipe UP (12 o'clock) — Centered Across Both (128x32)
  • Swipe DOWN-RIGHT (4 o'clock) — Duplicate (64x32 x2)
  • Swipe DOWN-LEFT (8 o'clock) — Mirror Right Side

Note: Want to rearrange which features are on which wheel, change their clock positions, or hide buttons you don't use? You can customize every single radial slot inside the Settings menu!

ProtoStream offers three distinct Display Modes that control how graphics are mapped onto your visor's two 64x32 LED matrices. You can switch between these at any time using the Display Mode Radial Menu on the main screen—even while you are actively streaming, and the visor will update on the fly!

• Duplicate (64x32 x2): This is the default setting. It takes a 64x32 image (or your scaled phone screen) and displays identical copies on both the left and right matrices.
• Centered Across Both (128x32): This mode spans your image continuously across both matrices, treating your visor as one wide, unified 128x32 pixel display split down the middle. This is ideal if you have artwork drawn on a 128x32 canvas or want asymmetrical expressions (like a wink or a raised eyebrow on only one side).
• Mirror Right Side: This name can sound slightly counter-intuitive at first, so here is how it works: it acts like Duplicate mode, except the image on the "right" matrix is flipped horizontally so both sides of your face stay symmetrical!
  (Important Geometry Note: "Right Side" refers to the panel on your right when the helmet is sitting on a table facing you. When you are actually wearing the helmet, the panel being flipped is on your anatomical left! If your eyes/mouth appear backward on your suit, your physical HUB75 panels were mounted or chained in the reverse order.)

At the bottom of the main screen, you'll find a collapsible card labeled "Static Image Slots". Tap anywhere on its top header bar to expand (▼) or collapse (▶) the drawer.

Unlike live screen streaming—which mirrors whatever is actively happening on your phone screen—Static Image mode streams a dedicated image or animated GIF directly from a background service. That means once you push an image or GIF from this menu, it will stay on your visor even if you exit the app, switch to other apps, or lock and turn off your phone's screen!

Note: GIFs will pause if you disconnect from the board's Wi-Fi.

When you expand the Static Image Slots drawer, you'll see two buttons across the top:

1. Non-Slot (Quick One-Time Upload):
   • Tapping "Non-Slot" opens your phone's gallery so you can pick any static image or animated GIF and immediately push it to your visor without saving it into a permanent slot.
   • Android will ask for permission to start screen capture/projection (this is required by Android to run the high-priority background streaming service)—be sure to accept it!
   • The uploaded image or GIF will automatically use whatever Display Mode and Interpolation mode you currently have selected.
2. + Add Slot (Persistent Custom Slots):
   • Tapping "+ Add Slot" creates a new permanent slot card in the list below. You can create as many slots as you'd like, and they will all be saved automatically when you close the app.
   • Here is how to use each slot card:
     • Uploading or Changing the Graphic: Tap directly on the square thumbnail box on the left side of the slot card to pick a static image or animated GIF from your device. (Note: The thumbnail is just a quick reference preview and does not reflect how the image will be scaled on the visor.)
     • Renaming a Slot: Quickly triple-tap (tap three times fast) directly on the slot's name text in the middle of the card! This unlocks the text box and pops up your keyboard so you can type a custom name.
     • Pushing to the Visor: Tap the blue circular ">" button on the right side of the card. Accept the Android screen-sharing prompt, and that slot's image or GIF will immediately start streaming to your visor!
     • Deleting a Slot: To prevent accidental deletions while suiting, the red delete button requires you to tap and HOLD it for 3 full seconds. A visual progress animation will fill up while you hold your finger down; once it finishes, the slot is removed.
     • Stopping a Static Stream: To pause a displayed GIF, simply use the Main Radial Menu and swipe down to "Stop Streaming / Clear".

Want to turn your visor into a reactive audio equalizer or display custom animated GIFs that groove to the beat of the music? That's where the Audio Visualizer comes in!

To open it, tap and hold the Main Radial button and swipe LEFT (9 o'clock). (If you were already streaming your screen, the app will automatically stop the screen stream for you before opening the visualizer!) The first time you open it, Android will prompt you for microphone access—select "While using the app" so the visualizer can hear surrounding music and voices.

This opens the "Visualizer Settings" pop-up menu, which gives you three main visual styles across the top:

• Solid: Displays classic frequency equalizer bars across your visor using a single solid color chosen from a preset list (Red, Green, Blue, Cyan, Magenta, Yellow, White, Orange, Purple, or Pink).
• Gradient: Lets you pick three distinct colors (Left, Middle, and Right) that blend together into a smooth horizontal color gradient across the equalizer bars.
• GIF: Lets you select a custom animated GIF that animates dynamically in response to sound! Tap "Select GIF" to pick your file. Beneath that, you'll find a toggle switch labeled "Rewind on Silence (off = Reset)":
  • With the toggle OFF (Default): The GIF plays forward while sound is detected, and immediately snaps back to Frame 1 the moment it gets quiet.
  • With the toggle ON: The GIF plays forward while sound is detected, and smoothly plays backward in reverse when silence is detected until it settles back at the first frame.

At the bottom of the pop-up, you'll find a slider for Microphone Sensitivity. Slide this higher in quiet rooms or lower at loud convention raves so the bars or GIF react cleanly without maxing out constantly. Note that GIF mode also inherits your active Display Mode and Interpolation setting!

Once you're happy with your settings, tap "Start" to launch the full-screen visualizer. While running, both Boop Detection and Head & Motion Gestures remain fully active, so people can still boop your snoot while you're dancing!

A Few Quick Survival Tips for the Visualizer:

• Bring a Portable Charger: Running real-time audio FFT processing at rave volume will give your battery a workout—keep a power bank in your pocket!
• How to Exit: Simply triple-tap anywhere on the black visualizer screen (or use your phone's back-swipe gesture) to close the visualizer and return to the main menu.

To open Vocal Sync, tap and hold the Main Radial button and swipe RIGHT (3 o'clock).

While Vocal Sync looks similar to the Emote Panel, it maps your expression slots directly to the volume of your voice in decibels (dB) rather than manual button presses! This lets your visor automatically animate mouth movements as you talk, or dynamically swap expressions based on how loudly you speak (for example, showing a normal talking mouth at conversational volume, and automatically switching to an intense or angry face when you yell!).

Here is how to set up and use Vocal Sync:

• The "Silent" Slot (0 dB): At the very top of your slot list, you will see a permanent default slot named "Silent" locked at 0 dB. This is your resting expression whenever you aren't speaking. To keep your baseline state unmistakable, the Silent slot cannot be deleted, renamed, or have its 0 dB threshold changed—just tap its "Upload" button to give it your default idle face or blinking GIF!
• Adding & Ordering Custom Slots: Tap the blue "Add Sync Slot" button to create a new voice-triggered slot (initialized at 50 dB by default). Every time you add or edit a slot's decibel level, the app automatically sorts all of your slots from top to bottom in order of lowest to highest volume threshold.
• Configuring Each Slot:
  • Upload: Tap "Upload" to assign a static image or animated GIF to that volume tier.
  • Rename: Tap "Rename" to give the slot a helpful label (like "Quiet Talk", "Normal Talk", or "Shouting").
  • Delete: Tap "Delete" and confirm the prompt to remove a custom slot.
  • Display Mode: Tap the mode button in the top-right corner of the slot card to cycle that specific slot between Duplicate (64x32 x2), Centered Across Both (128x32), and Mirror Right Side.
  • Set Activation Noise Level: Tap this green button to set the exact decibel (dB) volume required to trigger the slot. You can either type a number manually OR tap the "Auto Set" button and speak into your microphone at your target volume—the app will sample your voice for a couple of seconds and fill in the exact decibel number for you!
• Back-Tracking & Step-Down Delay: At the top of the Vocal Sync screen, you'll see a toggle switch labeled "Back-Tracking" with a "Settings" button next to it:
  • When Back-Tracking is OFF: Stopping your speech causes the visor to snap immediately from whatever loud slot was active straight back to the "Silent" (0 dB) slot.
  • When Back-Tracking is ON: When you stop talking, the visor smoothly steps backward down through each intermediate volume slot in sequence until it reaches "Silent", creating a natural mouth-closing transition! Tapping the "Settings" button next to the toggle lets you set the exact Back-Tracking Delay in milliseconds (ms) between each step.
• Launching & Running in the Background: Tap "Launch Vocal Sync" to start! A live decibel meter and reactive volume bar will appear right below the button, and the currently triggered slot will be highlighted with an "[ACTIVE]" tag. Built-in hardware noise suppression helps focus on your voice while filtering out background hum. Best of all, Vocal Sync continues running seamlessly even when you leave the screen or use your phone normally—just tap "Stop Vocal Sync" whenever you're ready to end the session.

Pro Tip: For the best responsiveness and isolation from crowd noise, I highly recommend pairing a small wireless clip-on Bluetooth microphone to your phone and mounting it inside your helmet right near your mouth!

To open the Emote Panel, tap and hold the Misc. Radial button (the middle button on the main screen) and swipe UP (12 o'clock).

The Emote Panel is your dedicated expression soundboard! It lets you hotswap between unlimited static images and animated GIFs instantaneously—without needing to start a screen capture stream at all—and includes full wireless controller support so you can change your face while wearing bulky hand-paws.

• Adding & Managing Emote Slots: Tap "Add Emote Slot" at the top of the page to create a new slot card. Each card includes:
  • Upload: Pick any static image or animated GIF from your device.
  • Rename: Change the slot's display name for easy reference.
  • Delete: Permanently remove the slot (with a confirmation prompt).
• Per-Slot Display Mode (Gear Icon): Tap the gear icon in the top-right corner of any emote slot to set its individual Display Mode (Centered Across Both (128x32), Duplicate (64x32 x2), or Mirror Right Side). If you change the Display Mode of a slot card while a GIF emote is actively playing, the visor updates immediately; for static images, simply tap the slot's thumbnail once more to refresh it.
• Controller Keybinds (Gamepad Icon): Right below the gear icon is a button with a controller icon. Tapping this opens the Controller Binding dialog:
  • Tap "Press to Listen for Input", then press any button, D-pad direction, bumper, trigger (LT/RT), or tilt either analog stick (Left Stick or Right Stick Up/Down/Left/Right) on your connected Bluetooth or USB controller!
  • Tap "Save" to lock in the binding (or "Clear Binding" to remove it).
  • As long as the Emote Panel screen is open, pressing that controller input will immediately push that emote straight to your visor! While it works with almost any standard gamepad, it is optimized and thoroughly tested for Xbox (X-Input) controllers.
• Touch Activation: Don't have a controller handy? You can also just tap directly on any emote slot's thumbnail preview to send it straight to your visor!

A Quick Note on GIF Performance: Because the Emote Panel streams raw decoded frames directly over UDP, extremely heavy or high-framerate GIFs may play back slightly slower if there is heavy Wi-Fi interference. If a complex GIF feels sluggish in the Emote Panel, try simplifying the GIF or upgrading to a (better) external uFL antenna on your MatrixPortal S3. I've found in testing that complex GIFs occasionally perform better when used as static images, but this may not always be the case.

To open the built-in pixel art studio, tap and hold the Misc. Radial button and swipe DOWN-LEFT (8 o'clock).

The Drawing suite lets you paint pixel art at your visor's exact native resolution! If your phone is currently connected to your Protogen's Wi-Fi, every single pixel stroke, flood fill, and edit you make on the canvas is streamed live to your physical visor in real time. Even if you aren't connected to your visor, the Drawing suite works 100% offline so you can design new faces anywhere.

• Dual Independent Canvas Modes (128x32 vs. 64x32):
  At the top-left of the screen, a toggle switch lets you flip between "128x32 Mode" and "64x32 Mode". Each mode acts as its own completely separate workspace with its own independent Undo/Redo history—meaning switching between 128x32 and 64x32 won't erase or overwrite what you were drawing in the other mode!
  • 128x32 Mode: Spans across both visor panels, divided down the center by a magenta guide line. Labels above each half ("Right (Anatomical)" and "Left (Anatomical)") remind you which side is which when wearing the helmet. Tapping the "Swap Sides" button in the top-right flips the left and right halves visually inside the app's editor if you prefer working from a front-facing perspective, without altering how the final art appears on the physical visor.
  • 64x32 Mode: Automatically duplicates your 64x32 canvas onto both sides of the visor ("Both Sides"), and disables the "Swap Sides" button.
• Pinch-to-Zoom & Two-Finger Pan:
  Working with tiny pixels on a phone screen is much easier when you can zoom in! Pinch with two fingers anywhere on the canvas to zoom in up to 10x for pixel-perfect detailing, and drag with two fingers to pan around your artwork.
• Drawing Tools & Color Picker:
  Below the canvas, you have three primary drawing tools plus a live color button:
  • Pencil (✏️): Draw freehand pixels and continuous lines with a single finger or stylus.
  • Paint Bucket (🪧): Flood-fill any enclosed area or background with your active color.
  • Eraser (🧼): Wipe individual pixels back to black (off).
  • Color Button (Colored Circle on the Right): Tap this circle to open the "Pick Color" studio! Inside, you can drag across the gradient box, type an exact Hex Code (e.g., #FF00FF), tap any of your 5 "Recent Colors" swatches, or tap "Pick Color From Canvas" to turn your finger into an eyedropper and sample any pixel directly off your drawing! Tap "Apply" to lock in your color.
• Full-Screen Landscape Mode:
  Need more room to draw? Tap the full-screen icon (⛶) in the top-right corner of the canvas border to rotate into a dedicated, edge-to-edge landscape view! While in full-screen mode, tap the small tab on the far-left edge of the screen to slide out your full tool drawer. Tap the corner icon again to return to portrait view.
• Canvas Management, Importing & 4 Ways to Save:
  • Undo & Redo: Step backward or forward through your stroke history (up to 30 steps per canvas mode).
  • Clear: Wipes the active canvas clean (protected by a confirmation prompt so you never clear by mistake).
  • Import Image: Brings any existing image from your phone onto the active canvas, automatically centering and scaling it to fit vertically while preserving its aspect ratio.
  • Save as Static Image Slot / Save as Vocal Sync Slot / Save as Emote Panel Slot: Tapping any of these three buttons prompts you to name your creation, then automatically creates a brand-new slot in that feature populated with your drawing and takes you straight there!
  • Save as Image: Exports your pixel art directly to your phone's photo gallery as a crisp PNG file for backup or external editing.

At the very top of Settings is the "Help & Tour Guide" card. Tapping this button opens a menu with Yen where you can either launch the complete onboarding tour from the beginning ("Give me the full tour!") or tap any of the 12 individual feature buttons (Connecting to Protogen, Main Radial Menu, Screen Streaming, Audio Visualizer, Vocal Sync, Misc Radial Menu, Emote Panel, Drawing Feature, Games Plugin Manager, Display Mode Radial Menu, Static Image Slots, or Settings Overview) to get an interactive guided walkthrough of just that specific tool.

For ease of access, ProtoStream automatically detects your phone's default language on first launch and supports 19 different languages—plus a secret 20th funny language—selectable at any time from the "Language" dropdown in Settings:

• English
• Simplified Chinese (简体中文)
• Traditional Chinese (繁體中文)
• Hindi (हिन्दी)
• Spanish (Español)
• French (Français)
• Arabic (العربية)
• Russian (Русский)
• Portuguese (Português)
• Indonesian (Bahasa Indonesia)
• German (Deutsch)
• Japanese (日本語)
• Turkish (Türkçe)
• Vietnamese (Tiếng Việt)
• Korean (한국어)
• Italian (Italiano)
• Thai (ไทย)
• Polish (Polski)
• Dutch (Nederlands)
• And a seekwit funni wanguage...

Note: Since I only fluently speak English, all other languages are translated and may contain occasional quirks. If you wish to help me improve the accuracy of any translation or want to help add your own language, feel free to reach out via the contact form at the top of this page!

Here you can update the Wi-Fi SSID (default: MatrixStreamer), Password (default: ProtoStream), and ESP32 Target IP Address (default: 192.168.4.1) that the app uses when connecting and streaming packets. Always tap the blue "Save" button at the bottom of the card after making changes! (Do not change the Target IP Address from 192.168.4.1 unless you are using the PC Emulator described below.)

Tapping the pink "Boop Detection" button in Settings opens a comprehensive setup panel that lets your Protogen react whenever someone boops your nose!

How Boops Are Detected:

• Hardware Light Sensor (Automatic): If you plugged an Adafruit VEML7700 Ambient Light Sensor into your MatrixPortal S3 and positioned it behind your visor near the tip of your nose, the ESP32 samples ambient light every 50ms. Whenever someone's hand or finger gets close to your snoot and casts a shadow over the sensor, ProtoStream detects the sudden drop in light and triggers a Boop!
• Controller Keybind (Manual): Don't have a light sensor installed, or want to trigger boop reactions yourself while wearing hand-paws? Under "Controller Keybind to Trigger a Boop", tap "Press to Listen for Input" and press any button, trigger, or stick direction on your controller. Pressing that input at any time will trigger a boop just like the physical sensor!
• Test Boop Button: At the top-right of the Boop Detection menu is a green "Test Boop" button. Tap this at any time to simulate a boop and test your setup live!

Boop Detection Controls & Multi-Boop Combo Tiers:

• Enable Boop Detection: Master toggle switch to turn boop reactions on or off globally.
• Light Sensor Sensitivity (0%–100%): Adjusts how sensitive the VEML7700 light sensor is to shadows. Higher percentages make even a light, quick hand wave trigger a boop; lower percentages require a firmer, closer cover of the sensor.
• Three Combinable Boop Action Modes: You can enable one, two, or all three of these switches at the exact same time to happen whenever you get booped:
  1. Image / GIF Overlay:
     Temporarily draws a custom reaction image or animated GIF right on top of whatever your visor is currently displaying (whether you're screen streaming, using Vocal Sync, in the Emote Panel, using the Audio Visualizer, or even just idling with no stream active!).
     • Overlay Opacity (0%–100%): Controls how transparent or solid the boop graphic appears over your current face.
     • Overlay Duration (ms): How long the boop graphic stays on your visor in milliseconds (default: 2000 ms, or 2 seconds). This also acts as your combo window for consecutive boops!
     • Boop Count Image / GIF Tiers (+ Add Boop Tier): Want your Protogen to get progressively more flustered—or annoyed—if someone keeps booping you over and over? You can create multiple Boop Tiers! Each tier has a "Boops in a row" number (e.g., 1, 2, 3, 5), its own "Display Mode" dropdown (Centered Across Both, Duplicate, or Mirror Right Side), and a "Select Image / GIF" button. If someone boops you repeatedly before the overlay duration expires, the combo counter climbs and automatically displays the matching tier's graphic!
     • Loop Back Toggle: Controls what happens when someone keeps booping you past your highest tier number. When Loop Back is ON, the counter wraps back around to Tier 1 and climbs up again; when OFF, it stays locked on your highest-numbered boop reaction until they stop booping you (unless they never stop booping you >:3c).
  2. Controller Keybind (Controller Keybind Sent When Booped):
     Lets a physical snoot boop simulate a controller button press inside the app! For example, you can bind this to an Emote Panel keybind or to Right Trigger (RT) so people at a convention can spin your ProtoSlots machine just by booping your nose!
  3. Launch Activity:
     Automatically opens a chosen screen in the app whenever you get booped (Main Screen, Audio Visualizer, Vocal Sync, Emote Panel, Drawing, Games Plugin Manager, or Custom Games).

Right below Boop Detection is the cyan "Head & Motion Gestures" button. Because the Adafruit MatrixPortal S3 has a built-in 3-axis accelerometer (LIS3DH), ProtoStream can track how your head moves in 3D space and trigger custom visor overlays, controller inputs, or app screens whenever you perform specific head movements (like tilting your head, nodding, shaking your head, or even sprinting!).

Step 1 — Calibrate Your Helmet's Mounting Angle:
Because every builder mounts their circuit board at a slightly different angle inside their head, the top of the Head & Motion Gestures menu features two quick calibration buttons (which also calibrate gravity for the ProtoPhysics game!):

• 🎯 Calibrate Flat Position: Put your head on, stand straight facing forward in your normal resting posture, and tap this button.
• 🔽 Calibrate Pointed Straight Down: Tilt your head/snout 90 degrees straight down toward the floor and tap this button. This teaches the app the exact 3D orientation of your visor!

Step 2 — Create & Record Gesture Slots:
Tap "+ Add Gesture Slot" to create a new custom gesture. Each slot is neatly housed in its own collapsible dropdown card with the slot's name displayed across the top header bar (along with a "✓" checkmark once a gesture has been recorded). Tap any slot's top bar to expand (▼) or collapse (▶) its settings:

• Rename, Test & Delete: At the top of the expanded slot, you can rename the gesture, tap the green "Test" button to fire its actions immediately, or delete the slot.
• Record Gesture (1.5-Second Motion Capture): Tap the purple "Record Gesture" button. It will turn orange and say "Recording (1.5s)... Move now!"—immediately perform your head movement (such as a sharp head tilt to the left, a double nod, or a head shake). The app automatically trims out any reaction-time delay at the start and end of the 1.5-second window and saves your 3D motion waveform!
• Hold Overlay While Motion Continues (Sprinting): When this switch is turned ON, your gesture overlay will stay active on your visor continuously for as long as you keep repeating the motion (perfect for sprinting, dancing, or headbanging animations!). When OFF, the gesture triggers once for its set duration followed by a brief cooldown so movements don't accidentally double-fire.
• Variance Threshold (1%–100%): Adjusts how forgiving the 3D gesture matcher is. If a gesture feels too strict to trigger, slide the Variance Threshold slightly higher; if it triggers too easily during normal head movement, slide it lower. I have found that ~80% is the sweet spot, but try out different thresholds and see what's right for you.
• Three Combinable Gesture Actions: Just like Boop Detection, each individual gesture slot has its own toggles for:
  1. Image / GIF Overlay (with its own custom image/GIF, Display Mode, Opacity slider from 0%–100%, and Duration in ms)
  2. Controller Keybind (sends a virtual controller button/trigger press when you perform the head gesture)
  3. Launch Activity (launches any chosen ProtoStream screen when you perform the gesture)

In Settings, the "Interpolation (Scaling Mode)" card lets you choose the mathematical algorithm used to scale your phone's screen or uploaded images down to the visor's 128x32 / 64x32 resolution. If you've used art programs like Photoshop, GIMP, or Aseprite, these will sound very familiar:

• None (1:1): Does not scale the image at all; instead, it crops the exact center 128x32 (or 64x32) pixels. Ideal if you are displaying native-resolution pixel art at 1:1 scale.
• Nearest-Neighbor: Scales images while preserving hard pixel edges with zero blurring. Ideal for pixel art and retro graphics that you want to stay crisp and blocky.
• Bilinear: Smooths out edges by blending neighboring pixels together. Ideal for photographs, smooth illustrations, or general screen streaming.
• Bicubic (High Quality): Uses a sharper multi-pixel sampling algorithm to preserve finer gradients and detail when scaling down high-resolution images.

At the bottom of Settings, you have complete control over how your main screen's radial menus behave:

• Show Display Mode Radial Menu: By default, this switch is ON, placing the third radial button (Display Mode) at the bottom of the main screen. If you turn this switch OFF, the third radial button is hidden from the main screen to save space, and a classic list of Display Mode radio buttons appears right inside this Settings card instead!
• Radial Menus Customization: This table lists all seven core tools (Start Live Screen Streaming, Stop Streaming / Clear, Audio Visualizer, Vocal Sync, Emote Panel, Games Plugin Manager, and Drawing). For each tool, you can use the two dropdowns to assign:
  • Menu: Choose whether the tool lives on the "Center" (Main) wheel, the "Misc" wheel, or set it to "Hidden" if you don't use it! (Note: Start Live Screen Streaming and Stop Streaming / Clear can only be moved between Center and Misc, and cannot be hidden.)
  • Position (1 to 12): Choose the exact clock-face position (1 = 1 o'clock, 3 = 3 o'clock/Right, 6 = 6 o'clock/Down, 9 = 9 o'clock/Left, 12 = 12 o'clock/Up) where that icon appears on the wheel! If you pick a position that another tool is already using on the same wheel, the two tools automatically swap places.
  • Reset to Default: Tap the red "Reset to Default" button at the bottom of the card to restore all radial buttons to their factory layout.

Note: This emulator script was not built with Linux in mind. I highly recommend using a Windows 10/11 PC instead if you have one available. If you are running the Python emulator in a Linux environment, you might run into some quirks with firewalls or display backends. Please note that I am not deeply familiar with Linux myself, so I likely will not be able to assist further if these steps do not resolve your issue!

Here are the common fixes if things act up on Linux:

• UDP Firewall Blocking (Port 2811): If the emulator window opens but no frames appear when streaming from your phone, your Linux firewall (UFW) may be blocking incoming UDP packets. Allow incoming traffic on port 2811 by running:

  sudo ufw allow 2811/udp

• Missing GUI Dependencies (libGL errors): If OpenCV complains about missing OpenGL libraries (like libGL.so.1), install the standard GUI packages on Debian/Ubuntu-based distributions via terminal:

  sudo apt update && sudo apt install libgl1 libglib2.0-0

• Display Server & X11/Wayland Issues: Ensure you are running an active desktop graphical session and that your DISPLAY environment variable is set. If you are running headless over SSH without X11 forwarding enabled, OpenCV won't be able to spawn the emulator window.
• Window Manager Freezing: The emulator script includes built-in socket timeouts to prevent GUI freezes when no packets arrive, but if your window manager still hangs, ensure your OpenCV package is up to date:

  pip install --upgrade opencv-python

Are you familiar with Java? ProtoStream includes a full Custom Games engine that lets you write your own 128x32 visor games in Java, compile them into portable .protogame files, and share them with anyone else who uses ProtoStream!

• How to Import & Play: Inside the Games Plugin Manager, tap the purple "Custom Games" button near the top of the screen. Next, tap "Import .protogame" and select any .protogame file on your phone. Once imported, tap "Launch" to play!
• Built-In Touch & Physical Controller Support: When you launch a Custom Game, ProtoStream opens a dedicated host screen displaying a live preview of the 128x32 visor at the top and a built-in on-screen touch controller underneath (D-Pad Up/Down/Left/Right, Select, Start, B, A, and an Exit button). If you have a wireless Xbox/Bluetooth controller or keyboard connected, the host automatically maps your physical D-pad/left stick and buttons to the game as well!
• Build Your Own: Interested in making your own visor game? Check out the official ProtoStream Dev Kit and documentation on GitHub.

ProtoTetris is a custom-built block puzzle game tailored specifically for the visor's 128x32 dual-matrix layout! The game splits your visor into two halves:

• Left Visor Panel: Displays the main playfield where you maneuver and stack falling tetrominoes.
• Right Visor Panel: Displays your current score, a preview of the NEXT tetromino, and your currently HELD tetromino.

ProtoTetris requires a wireless or wired Xbox/Bluetooth controller connected to your phone. It features two play orientation modes: VERT (Top-Down) and HORZ (Left-to-Right), which you can select on the title screen using the Left Stick or D-Pad before pressing A to start:

• Left Stick / D-Pad: Move the falling tetromino left/right (or up/down in HORZ mode) and soft-drop.
• A or B Button: Rotate the tetromino clockwise.
• X or Y Button: Swap the current tetromino into the Hold slot.
• Hard Drop: In VERT mode, press Up on the Left Stick or D-Pad to instantly drop and lock the piece at the bottom. In HORZ mode, press Left on the Left Stick or D-Pad to instantly drop the piece to the far side of the board.
• Menu Button (☰): End the current run and return to the mode selection title screen.

Your high score is saved automatically on your device, so you can always try to beat your personal best or hand your controller to friends at conventions to challenge your record! To exit ProtoTetris and return to the Games Plugin Manager, simply triple-tap anywhere on your phone screen (or use your phone's native back-swipe gesture).

ProtoFish is a complete, custom-built fishing RPG designed specifically for the visor's 128x32 resolution! Cast your line, battle fish with directional rod tension, earn cash, upgrade your gear, travel to exotic ponds, and complete a massive 96-entry Fish Index!

I want to give a special thanks to Xevi for creating many of ProtoFish's assets.

Like ProtoTetris, ProtoFish is played using a wireless or wired Xbox/Bluetooth controller paired to your phone. To exit the game and return to the Games Plugin Manager at any time, simply triple-tap anywhere on your phone screen.

Who doesn't like a little convention gambling? ProtoSlots is a customizable slot machine game built for your visor—perfect for running interactive prize giveaways at conventions or just spinning for fun!

Unlike ProtoTetris and ProtoFish, ProtoSlots can be played EITHER with a connected Xbox controller OR simply by tapping directly on your phone screen!

ProtoPhysics comes pre-installed right inside the Games Plugin Manager—no separate plugin download required! It turns your 128x32 visor into a live, interactive falling-sand and particle physics simulation driven by the 3-axis accelerometer built into your Adafruit MatrixPortal S3 (or your phone's internal tilt sensors if you are testing without an ESP32).

As you tilt your head left, right, up, or down, hundreds of glowing pixels pour, stack, and slide across both sides of your visor in real time like digital neon sand!