docs: clean up, yeet README

50fe7b7e · q3k · q3k · 8b37dbbe · 50fe7b7e · 50fe7b7e
Commit 50fe7b7e authored 1 year ago by q3k Committed by q3k 1 year ago
--- a/README.md
+++ b/README.md
-## Demo Payload
+# flow3r badge
-See python_payload/README.md for a demo application.
+See https://docs.flow3r.garden/badge/firmware-development.html for a firmware development guide.
-Files can be transferred either by performing an `idf.py flash` or with mpremote, such as:
-```
-mpremote fs cp python_payload/main.py :main.py
-```
-Alternatively, adafruit-ampy may work more reliably sometimes:
-```
-ampy -p /dev/ttyACM0 -d3 put main.py
-```
-Please transfer all .py files in python_payload/ for using the demo payload.
-## How to install dependencies
-Pay attention to install version 5.1 **WITH OUR PATCHES**. To be sure follow the generic instructions below.
-### Generic
-1. install esp-idf v5.1:
-(copied from https://www.wemos.cc/en/latest/tutorials/others/build\_micropython\_esp32.html)
-```
-$ cd ~
-$ git clone https://github.com/espressif/esp-idf.git
-$ cd esp-idf
-$ git checkout v5.1
-$ git submodule update --init --recursive
-$ patch -p1 < PathToYourFlow3rFirmwareRepo/third_party/b03c8912c73fa59061d97a2f5fd5acddcc3fa356.patch
-$ ./install.sh
-$ source export.sh
-```
-best put something like "alias espidf='source ~/esp-idf/export.sh'" in your .bashrc etc,
-you need to run it in every new terminal and adding it to autostart did bother us
-Alternatively, we have a fork of esp-idf in this gitlab (TODO: update to new
-gitlab url and new org name when that's a thing)
-```
-$ cd ~
-$ git clone git@git.card10.badge.events.ccc.de:badge23/esp-idf.git
-$ cd esp-idf
-$ git checkout v5.1-flow3r
-$ git submodule update --init --recursive
-```
-### Nix(OS)
-```
-$ nix-shell nix/shell.nix
-```
-## How to build and flash
-Standard ESP-IDF project machinery present and working. You can run `idf.py` from the git checkout and things should just work.
-### Building
-Prepare submodules:
-```
-git submodule update --init --recursive
-```
-Build normally with idf.py:
-```
-$ idf.py build
-```
-By default, code for the fourth generation prototype will be built. To select a different generation, either set `-g`/`--generation` during an `idf.py build` (which will get cached for subsequent builds) or set the BADGE_GENERATION environment variable to one of the following values:
-| `-g` / `BADGE_GENERATION` value | Badge Generation                   | 
-|---------------------------------|------------------------------------|
-| `p1` or `proto1`                | Prototype 1                        |
-| `p3` or `proto3`                | Prototype 3 (B3xx)                 |
-| `p4` or `proto4`                | Prototype 4 (B4xx)                 |
-| `p6` or `proto6`                | Prototype 6 (B6xx)                 |
-**Important**: when switching generations, do a full clean by running `rm -rf sdkconfig build`. Otherwise you will get _weird_ errors and likely will end up building for the wrong architecture.
-### Flashing
-Put badge into bootloader mode by holding left should button down during boot.
-```
-$ idf.py -p /dev/ttyACM0 flash
-```
-The following targets are available for flashing:
-| Target             | Flashes                             |
-|--------------------|-------------------------------------|
-| `idf.py flash`     | Bootloader, partition table, C code |
-| `idf.py app-flash` | C code                              |
-You can skip `-p /dev/ttyACM0` if you set the environment variable `ESPPORT=/dev/ttyACM0`. This environment variable is also set by default when using Nix.
-After flashing, remember to powercycle your badge to get it into the user application.
-### Cleaning
-For a full clean, do **not** trust `idf.py clean` or `idf.py fullclean`. Instead, do:
-```
-$ rm -rf build sdkconfig
-```
-### Accessing MicroPython REPL:
-```
-$ picocom -b 115200 /dev/ttyACM0
-$ # or
-$ screen /dev/ttyACM0
-$ # or (will eat newlines in REPL, though)
-$ idf.py -p /dev/ttyACM0 monitor
-$ # or
-$ mpremote repl /dev/ttyACM0
-```
-and press Ctrl-C (potentially twice) to exit the debug output.
-### Use CMake
-`idf.py` calls cmake under the hood for most operations. If you dislike using wrappers you can do the same work yourself:
-```
-mkdir build
-cd build
-cmake .. -G Ninja
-ninja
-```
-There's `flash/monitor` targets, too (but no openocd/gdb...). To pick what port to flash to/monitor, set the ESPPORT environment variable.
-## How to modify
-### Structure
-```
-components/flow3r_bsp/             - Board Support Package, ie. low-level drivers
-components/st3m/                   - Core C software (C-st3m)
-python_payload/st3m/               - Core Python software (Py-st3m)
-components/micropython/usermodule/ - Bindings between C-st3m and Py-st3m.
-components/micropython/vendor/     - Micrpython fork.
-```
-### General info
-Global + micropython entry point: `app_main()` in `micropython/ports/esp32/main.c`, compiled into `main/` component.
-C entry point, called by above: `st3m_board_init()` in `components/st3m/st3m_board_init.c`
-After C-st3m initializes, it returns and lets Micropython run. Micropython then will stay in a loop which attempts to run main.py, otherwise runs the REPL.
-### Debugging
-The badge starts with a UART/JTAG bridge, but then after boot switches to a custom USB stack based on TinyUSB. This stack will bring up a serial-based console that will run the Micropython REPL.
-The serial console will carry any C printf() you throw in, and any active `ESP_LOGx` call (see: ESP logging levels in sdkconfig). It will also carry anything that micropython prints.
-However, if the badge crashes, you will not see any output on the console. You are also not able to run OpenOCD/gdb. If you wish to perform these actions, you will have to modify `st3m_board_init.c` to disable USB console startup. You can also switch the default console to UART0 and use the UART0 peripheral over USB-C sideband pins.
-See [tracking issue](https://git.card10.badge.events.ccc.de/badge23/firmware/-/issues/23).
-#### printf() debugging and logging in C-land
-Given the above, you can do the following to get a log. This is part of ESP-IDF's standard logging functionality.
-```
-static const char *TAG = "misery";
-// ...
-ESP_LOGI(TAG, "i love C");
-```
-However, this will **only work** if you first set `CONFIG_LOG_DEFAULT_LEVEL_INFO=y` (which will likely break programs interacting with micropython REPL as many components of the ESP-IDF will suddenly become very chatty). But that's fine for troubleshooting some C-land bugs.
-If you want to only log errors or just add temporary logs, use `ESP_LOGE` instead, which will always print to the USB console.
-`printf()` also just works. But it should only be used for development debugging, for long-term logging statements please use `ESP_LOG*` instead.
-#### Running OpenOCD+GDB
-(currently broken, see [tracking issue](https://git.card10.badge.events.ccc.de/badge23/firmware/-/issues/23).
-First, make sure your badge is running in application mode (not bootloader mode! that will stay in bootloader mode).
-Then, start OpenOCD:
-```
-$ OPENOCD_COMMANDS="-f board/esp32s3-builtin.cfg" idf.py openocd
-```
-(you can skip setting `OPENOCD_COMMANDS` if you're using Nix)
-Then, in another terminal:
-```
-$ idf.py gdb
-```
-Good luck. The idf.py gdb/openocd scripts seem somewhat buggy.
-### ESP-IDF functionality
-#### sdkconfig / menuconfig
-We have an sdkconfig.defaults file. It is used to generate a '.generated' file
-with flow3r-specific options based on the given -g option. See the build
-instructions above to see how to select the generation to build against.
-The build system will generate an sdkconfig, but it should not be committed
-into version control. Instead, treat it like an ephemeral artifact that you can
-also modify for your own needs during development.
-To run menuconfig, do the usual::
-```
-$ idf.py  menuconfig
-```
-(Specify -g or `BADGE_GENERATION` if you haven't built the firmware yet)
-Then, either save into the temporary sdkconfig by using 'S', or save into a
-defconfig by using 'D'. The resulting `build/defconfig` file can then be copied
-into `sdkconfig` to change the defaults for a given generation (be sure to
-remove FLOW3R_* options that are usually generated by `idf_ext.py`).
-### Documentation
-Automatically updated on CI runs of the main branch and lives under https://docs.flow3r.garden
-#### Build
-To build sphinx docs:
-```
-cd docs
-make html
-firefox _build/html/index.html
-```
-To continuously build on change:
-```
-watchexec make html
-```
 ## License
 All original source code in this reporitory is Copyright (C) 2023 Flow3r Badge Contributors. This source code is licensed under the GNU General Public License Version 3.0 as described in the file COPYING.

--- a/docs/api/ctx.rst
+++ b/docs/api/ctx.rst
 ``ctx`` module
 ==============
-.. automodule:: st4m.ui.ctx
+.. automodule:: st3m.ui.ctx
   :members:
   :undoc-members:
--- a/docs/badge/firmware-development.rst
+++ b/docs/badge/firmware-development.rst
@@ -14,7 +14,7 @@ firmware repository.
 ::
-	$ git clone TODO
+	$ git clone https://git.flow3r.garden/flow3r/flow3r-firmware
 Dependencies
 ------------
@@ -25,12 +25,10 @@ On other Linux-based distributions, you will have to manually install ESP-IDF al
 ::
-	$ git clone https://github.com/espressif/esp-idf.git
+	$ git clone https://git.flow3r.garden/flow3r/esp-idf
 	$ cd esp-idf
-	$ git checkout v5.1
+	$ git checkout v5.1-flow3r
 	$ git submodule update --init --recursive
-	$ cd esp-idf
-	$ patch -p1 < ../firmware/third_party/b03c8912c73fa59061d97a2f5fd5acddcc3fa356.patch
 	$ ./install.sh
 	$ source export.sh
@@ -59,7 +57,7 @@ You can use `mpremote` and similar to copy edited files from ``python_payload/``
 	$ mpremote cp python_payload/main.py :/flash/sys/main.py
-*TODO: document mpremote mount*
+*TODO: document mpremote mount, it's currently broken*
 As with application development, you can first check your changes using the simulator:
@@ -82,18 +80,14 @@ To compile:
 	$ idf.py build
-To flash, put the badge in :ref:`flash` mode and run:
+To flash the main firmware only (without overwriting the FAT32 partition or recovery image), put the badge in :ref:`flash` mode and run:
-::
-	$ idf.py flash
-Or, to just write the main firmware partition:
 ::
 	$ idf.py app-flash
+Note: do not run ``idf.py flash`` as that will prevent you from going into recovery mode. If you're flashing a factory-new badge, you also need to flash the recovery partition/bootloader/firmware first. See `flashing recovery`_.
 To clean, do not trust ``idf.py clean``. Instead, kill everything with fire:
 ::
@@ -108,6 +102,50 @@ To edit the sdkconfig temporarily:
 To commit your sdkconfig changes to git, run menuconfig, press *d*, accept the default path. Then, copy over ``build/defconfig`` onto ``sdkconfig.defaults``.
+.. _`flashing recovery`:
+Flashing Recovery
+-----------------
+Tl;DR use the following script to flash *everything*:
+::
+	$ tools/flash-full.sh
+The long story is that the main firmware codebase has a slightly different
+partition layout (as seen by the flashing tooling) than the recovery tooling.
+The one used in the recovery project (``recovery/partitions.csv``) is the
+correct one. However, we can't use it as the main ``partitions.csv`` file as
+ESP-IDF performs magical detection from that file on where the build artifact
+should be located, and it always defaults to flashing to the ``factory`` image.
+Thus, in the real/recovery partition table the recovery firmware is the
+``factory`` image, while the main firmware is in the ``ota_0`` partition. But to
+make ``idf.py app-flash`` work in the main firmware repository, there the main
+firmware is marked as ``factory``. But if you flash the main firmware's
+partition table to the device, the recovery partition will stop working.
+In addition to Different-Partition-Table shenanigans, the second-stage
+bootloader is also a problem. As with the partition teable, the correct one is
+the recovery one. Using this bootloader allows you to pick the recovery image on
+startup by holding the right trigger.
+So, in order to have a functioning badge you shoud:
+ 1. Flash the partition table from recovery
+ 2. Flash the bootloader from recovery
+ 3. Flash the factory image from recovery
+ 4. Flash the ota_0 image from main
+Or, in code:
+::
+	$ (cd recovery && idf.py erase-flash flash)
+	$ idf.py app-flash
+Thich is what ``tools/flash-full`` does.
 printf-Debugging
 ----------------
@@ -133,3 +171,45 @@ You should stay compatible with our :ref:`partition` layout. The easiest way to
 Then, you can run your firmware by distributing the resulting ``.bin`` file and letting people flash to it via :ref:`Recovery Mode`.
 For an example, see our doom port at **TODO**.
+Hardware Generations
+--------------------
+If you've received your badge at CCCamp2023, you have a Production Badge and thus you don't need to worry about this section. Congratulations!
+For those who have a prototype badge, there's an ``idf.py -g pX`` flag which you can use to get the firmware running on your hardware:
+------------------+----------+-----------------------------------+
+| Badge Generation | Markings | Flag                              |
+==================+==========+===================================+
+| Prototype 4      | B4xx     | *dead*                            |
+------------------+----------+-----------------------------------+
+| Prototype 3      | B3xx     | ``-g p3``                         |
+------------------+----------+-----------------------------------+
+| Prototype 4      | B4xx     | ``-g p4``                         |
+------------------+----------+-----------------------------------+
+| Prototype 5      | B5xx     | *port me*                         |
+------------------+----------+-----------------------------------+
+| Prototype 6      | B6xx     | ``-g p6`` (default, same as prod) |
+------------------+----------+-----------------------------------+
+*NOTE: Anything older than p6 is not (yet?) supported by the recovery firmware.*
+Writing Docs
+------------
+Automatically updated on CI runs of the main branch and lives under https://docs.flow3r.garden.
+To build the docs locally:
+::
+    $ cd docs
+    $ make html
+    $ firefox _build/html/index.html
+To continuously build on change:
+::
+    $ watchexec make html
--- a/docs/badge/firmware.rst
+++ b/docs/badge/firmware.rst
@@ -83,19 +83,19 @@ If the badge is running correctly, you can access the filesystem over the microp
 ::
 	$ mpremote
-	MicroPython c48f94151-dirty on 1980-01-01; badge23 with ESP32S3
+	MicroPython c48f94151-dirty on 1980-01-01; flow3r with ESP32S3
 	Type "help()" for more information.
 	>>> import os
 	>>> os.listdir('/')
 	['flash']
 	>>> os.listdir('/flash/sys')
-	['main.py', 'st4m', '.sys-installed']
+	['main.py', 'st3m', '.sys-installed']
 	>>> 
 	$ mpremote ls :flash/sys
 	ls :flash/sys
 	           0 main.py
-	           0 st4m
+	           0 st3m
 	           0 .sys-installed

--- a/docs/badge/programming.rst
+++ b/docs/badge/programming.rst
@@ -102,19 +102,19 @@ You can also access the filesystem over the same Micropython serial port:
 ::
 	$ mpremote
-	MicroPython c48f94151-dirty on 1980-01-01; badge23 with ESP32S3
+	MicroPython c48f94151-dirty on 1980-01-01; flow3r with ESP32S3
 	Type "help()" for more information.
 	>>> import os
 	>>> os.listdir('/')
 	['flash']
 	>>> os.listdir('/flash/sys')
-	['main.py', 'st4m', '.sys-installed']
+	['main.py', 'st3m', '.sys-installed']
 	>>> 
 	$ mpremote ls :flash/sys
 	ls :flash/sys
 	           0 main.py
-	           0 st4m
+	           0 st3m
 	           0 .sys-installed
 .. _disk mode: