docs/app/api/captouch.rst

+``captouch`` module
+===================
+
+Basic usage
+-----------
+
+In a flow3r application you receive a ``CaptouchState`` object in each ``think()`` cycle. Here's a simple example:
+
+.. code-block:: python
+
+    class App(Application):
+        def think(self, ins, delta_ms):
+            petal_0_is_pressed = ins.captouch.petals[0].pressed
+
+You cannot instantiate this object directly, but for REPL experiments there is a workaround listed below.
+
+.. py:class:: CaptouchState
+
+    .. py:attribute:: petals
+        :type: Tuple[CaptouchPetalState]
+
+        State of individual petals.
+
+        Contains 10 elements, with the zeroeth element being the petal closest to
+        the USB port. Then, every other petal in a clockwise direction.
+
+        Even indices are top petals, odd indices are bottom petals.
+
+        The top petal indices are printed in roman numerals around the flow3r display,
+        with "X" corresponding to 0.
+
+    .. py:attribute:: ticks_us
+        :type: int
+
+        Timestamp of when the captouch data has been requested from the backend, i.e. when
+        the ``think()`` cycle started. Mostly useful for comparing for to the same attribute
+        of ``PetalLogFrame``. Behaves identical to the return type of ``time.ticks_us()``
+        and should only be used with ``time.ticks_diff()`` to avoid overflow issues. Overflow
+        occurs after ~10min.
+
+
+.. py:class:: CaptouchPetalState
+
+    .. py:attribute:: pressed
+        :type: bool
+
+        True if the petal has been touched during the last ``think()`` cycle.
+
+        May be affected by ``captouch.Config``.
+
+    .. py:attribute:: pos
+        :type: Optional(float)
+
+        Coordinates where this petal is touched or None if the petal isn't
+        touched or positional output is turned off via `captouch.Config``.
+
+        The coordinate system is rotated with the petal's orientation: The
+        real part corresponds to the axis going from the center of the screen
+        to the center of this petal, the imaginary part is perpendicular to
+        that so that it increases with clockwise motion.
+
+        Both real and imaginary part are centered around 0 and scaled to a
+        [-1..1] range. We try to guarantee that the output can span the full
+        unit circle range, but it may also go beyond.
+
+        Some filtering is applied.
+
+        May be affected by ``captouch.Config``.
+
+        See ``captouch.PETAL_ROTORS`` to align the output with the display
+        coordinate system.
+
+    .. py:attribute:: raw_pos
+        :type: float
+
+        Similar to ``.pos``, but never None. Will probably return garbage when
+        petal is not pressed. It is mostly useful for interpolating data between
+        petals. Filtering is still applied.
+
+    .. py:attribute:: raw_cap
+        :type: float
+
+        Returns a the raw capacity reading from the petal in arbitrary units.
+        The value that kind-of-sort-of corresponds to how much the pad
+        is covered. Since the footprint of a finger expands when compressed
+        (what a sentence), this could in theory be roughly used for pressure,
+        but the data quality just doesn't cut it:
+
+        It's mostly okay when not compared against fixed values, but rather
+        some sort of floating average, but it's not really monotonic and also
+        it doesn't react until the finger is a few mm away from the pad so it's
+        kinda bad for proximity sensing too. It's tempting to use it for gating
+        away light touches, but that results in poor performance in some
+        environmental conditions. Test carefully, and best make nothing important
+        depend on it.
+
+        Normalized so that "1" corresponds to the upper hysteresis limit of the
+        ``pressed`` API.
+
+        May be affected by ``captouch.Config``.
+
+    .. py:attribute:: log
+        :type: Tuple[PetalLogFrame]
+
+        Raw frame output of the captouch driver. Must be enabled by ``captouch.Config``.
+
+        Since micropython and the captouch driver are running asynchronously we're providing
+        a list of all raw data points collected since the last ``.think()`` call.
+
+        The lowest indices are the oldest frames, so that you could compile a complete log
+        (or one cropped to arbitrary length) simply by appending new data:
+
+        .. code-block:: python
+
+            def on_enter(self, vm):
+                super().on_enter(vm)
+                conf = captouch.Config.default()
+                conf.petals[0].logging = True
+                conf.apply()
+                self.log = list()
+
+            def think(self, ins, delta_ms):
+                super().think(ins, delta_ms)
+                # append new frames to end of log
+                self.log += ins.captouch.petals[0].log
+                # crop old frames
+                self.log = self.log[-100:]
+
+.. py:data:: PETAL_ROTORS
+   :type: Tuple[complex]
+ 
+    Tuple of 10 constants that can be used to rotate the output of the `.pos` attribute of both `CaptouchPetalState` and
+    `PetalLogFrame` to align with the display (x, y) coordinates.
+
+    .. code-block:: python
+
+        # (in think)
+        for x in range(10):
+            pos = ins.captouch.petals[x].pos
+            pos *= 60 * captouch.PETAL_ROTORS[x]
+            self.display_coords[x] = (pos.real, pos.imag)
+
+        # (in draw)
+        for x in range(10):
+            ctx.move_to(* self.display_coords[x])
+            ctx.text(str(x))
+
+.. py:data:: PETAL_ANGLES
+   :type: Tuple[float]
+ 
+    Tuple of 10 constants that can be used to align with the display (x, y) coordinates. ``PETAL_ANGLES[x]`` is equivalent
+    to ``cmath.phase(PETAL_ROTORS[x])`` and ``PETAL_ROTORS[x]`` is equivalent to ``cmath.rect(1, PETAL_ANGLES[x])``.
+
+Speeding things up
+------------------
+
+The flow3r captouch driver is not the fastest, it takes at least 14ms to generate a full dataset with all channels running.
+For applications where speed is key it is possible to merge data channels to reduce scanning time. Each petal can be turned
+off entirely, most can act as simple a button or a 1D slider (also 2D for all top petals). For example, if you turn off all
+petals except for 2 and 8 for a "dual joystick" mode, you increase your frame rate to up to 2.3ms!
+
+.. code-block:: python
+
+    import captouch
+
+    class App(Application):
+        def __init__(self):
+            self.captouch_config = captouch.Config.empty()
+            # top petals are used as buttons, bottom petals not at all
+            for petal in range(0,10,2):
+                self.captouch_config.petals[petal].mode = 1
+
+        def on_enter(self, vm):
+            self.captouch_config.apply()
+
+.. py:class:: Config
+
+    .. py:method:: empty() -> Config:
+        :classmethod:
+
+        Initializer method that returns a config with everything disabled. Ideal
+        for ORing the requirements of different components together.
+
+    .. py:method:: default() -> Config:
+        :classmethod:
+
+        Initializer method that returns the default config, same as when entering an application.
+
+    .. py:method:: current() -> Config:
+        :classmethod:
+
+        Initializer method that returns the currently active config.
+
+    .. py:method:: apply() -> None:
+        
+        Apply this config to the driver.
+
+    .. py:method:: apply_default() -> None:
+
+        Convenience method to restore defaults. same as ``Config.default().apply()`` but mildly faster
+        if you already have one around.
+
+    .. py:attribute:: petals
+        :type: Tuple[PetalConfig]
+
+        Config of individual petals, indexed as in the ``CaptouchState`` object.
+
+.. py:class:: Config.PetalConfig
+
+    .. py:attribute:: mode
+        :type: int
+
+        What kind of data should be collected for this petal. Raises ``ValueError`` when set to an unallowed value.
+
+        0: No data at all. Allowed for all petals.
+
+        1: Button Mode: All pads combined, no positional output. Only allowed for bottom petals and petals 4 and 6.
+
+        2: 1D: Only radial position is provided. Only allowed for bottom petals and petals 4 and 6.
+
+        3: 2D: Full positional output. Only allowed for top petals.
+
+        Defaults to the maximum allowed value.
+
+        The integer value corresponds to the number of active chip channels. Data rate scales linearily per chip
+        at 0.75ms per channel plus a noisy overhead of 2-4ms typically. Bottom petals and petal 2 are connected to
+        one chip, the remaining top petals to another.
+
+        *Note: We discovered last-minute that modes 1 and 2 are not functioning properly for some top petals, so
+        they are currently unavailable. We will try to fix them up in the future. They work fine for petals 4 and
+        6 due to their lower bulk capacity, presumably because of the speaker holes.*
+
+    .. py:attribute:: logging
+        :type: bool
+
+        Whether or not you want to collect the raw data log. This
+        eats some CPU time proportional to the ``think()`` cycle
+        time, use only when you actually do something with the data.
+
+        Default: False
+
+    .. py:method:: set_min_mode(mode: int) -> None:
+
+        If the current mode is lower than the argument, it gets increased to that value if allowed. If the value
+        is not allowed it is either set to the next-biggest allowed value or, if no such value exists, to the
+        largest allowed value.
+
+
+Gestures
+--------
+
+For many common applications we provide widgets that do whatever data processing is needed so you don't have to implement
+everything from scratch, see ``st3m.ui.widgets``. If whatever you want is already in there we recommend using these as
+future performance improvements will then directly benefit your application.
+
+If you do want to do your own signal processing you will probably want to use the logging feature: The positional data is
+fairly imperfect already, missing frames or needing to detect duplicates doesn't make it better, and the general-purpose
+filtering on the "primitive" positional output may be an issue for fast motion detection. Using the unprocessed log
+doesn't make postprocessing easy, but at least you get the best data quality the driver can offer.
+
+In order to use the logging feature effectively we also provide a ``PetalLog`` class that implements fast time-based
+cropping and data processing. This could all be done in raw python too, but e.g. for linear regression the C implementation
+runs around 40 times faster and creates less intermediate objects so that garbage collection triggers less. This is
+particularily important if the captouch driver is configured to run a specific petal very fast.
+
+.. py:class:: PetalLogFrame
+
+    .. py:attribute:: pressed
+        :type: bool
+
+        Identical to ``pressed`` of ``CaptouchPetalState``.
+
+    .. py:attribute:: pos
+        :type: Optional(float)
+
+        Identical to ``pos`` of ``CaptouchPetalState`` but without any filtering.
+
+    .. py:attribute:: raw_pos
+        :type: float
+
+        Identical to ``raw_pos`` of ``CaptouchPetalState`` but without any filtering.
+
+    .. py:attribute:: raw_cap
+        :type: float
+
+        Identical to ``raw_cap`` of ``CaptouchPetalState``.
+
+    .. py:attribute:: mode
+        :type: int
+
+        Config mode setting that was used for recording the frame (see ``captouch.Config``).
+
+    .. py:attribute:: ticks_us
+        :type: int
+
+        Timestamp that reflects the approximate time at which the data was captured (to
+        be exact, when the I2C transmission has completed). Behaves identical to the return
+        type of ``time.ticks_us()`` and should only be used with ``time.ticks_diff()``
+        to avoid overflow issues. Overflow occurs after ~10min.
+
+
+.. py:class:: PetalLog
+
+    .. py:attribute:: frames
+
+        List of PetalLogFrames. May be manipulated or replaced by user. We use the binary structure of micropython
+        ``list`` as well as ``PetalLogFrame``, so any duck typing may result in ``TypeError`` when the other attributes
+        and methods of this class are used.
+
+    .. py:method:: append(frame: PetalLogFrame):
+    
+        Appends frame to ``.frames``. There's a performance benefit when only modifying ``.frames`` with this method
+        alongside ``.crop()`` and ``.clear()``.
+
+    .. py:method:: crop(index: Optional(int)) -> int
+
+        Crops the oldest elements in ``.frames`` in-place and returns the number of cropped frames. The ``index``
+        parameter behaves slice-like, equivalent to ``.frames = .frames[index:]``, i.e. positive values remove
+        that amount of oldest frames, negative values limit the list at most ``-index`` frames and None does nothing.
+        Typically used together with ``index_offset_ms()`` to keep the length of ``frames`` in check.
+
+    .. py:method:: clear()
+
+        Clears ``.frames``.
+
+    .. py:method:: length() -> int
+
+        Returns ``len(.frames)`` but slightly faster.
+
+    .. py:method:: length_ms(start: Optional(int) = None, stop: Optional(int) = None, /) -> float
+
+        Returns difference in timestamp between newest and oldest frame in milliseconds or 0 if ``.frames`` is empty.
+        The optional ``start`` and ``stop`` parameters delimit which slice of ``.frames`` is used for computation,
+        equivalent to ``.frames[start:stop]``. Negative values behave as expected.
+
+    .. py:method:: index_offset_ms(index: int, min_offset_ms: float, /) -> Optional(int)
+
+        Returns the index of the frame that is at least ``min_offset_ms`` newer (or older for negative ``min_offset_ms``)
+        than the frame at ``index``, or ``None`` if no such frame exists. Negative ``index`` values are allowed and work
+        as expected, e.g. ``index = -1`` indicates the newest frame. Will raise ``IndexError`` if the index is out of range.
+
+    .. py:method:: average(start: Optional(int) = None, stop: Optional(int) = None, /) -> Optional(complex)
+
+        Returns the average position of elements in ``.frames``. Will return ``None`` if no frames are available.
+        The optional ``start`` and ``stop`` parameters delimit which slice of ``.frames`` is used for computation,
+        equivalent to ``.frames[start:stop]``. Negative values behave as expected.
+
+    .. py:method:: slope_per_ms(start: Optional(int) = None, stop: Optional(int) = None, /) -> Optional(complex)
+
+        Returns the ordinary least squares linear regression slope of the position of elements in ``.frames``. Uses
+        timestamp and disregards order of ``.frames``. Will return ``None`` if less than 2 frames are available or all
+        timestamps are equal.
+        The optional ``start`` and ``stop`` parameters delimit which slice of ``.frames`` is used for computation,
+        equivalent to ``.frames[start:stop]``. Negative values behave as expected.
+
+
+The nitty gritty
+----------------
+
+The flow3r captouch setup is not as good as a smartphone touchscreen. While a typical modern touchscreen receives data from
+a fine grid of wire intersections, flow3r just has 2 per bottom pad and 3 per top pad. Here's an illustration:
+
+.. image:: assets/captouch_petals.png
+
+On a grid type touch device you can infer rough position even with rather high noise levels as long as a "high" and "low"
+for each grid point is roughly represented. On a device like flow3r, unfortunately we do not have this luxury. This leads
+to higher noise sensitivity and some other unexpected behaviors that limit how captouch can be used:
+
+**Liftoff artifacts**
+
+In general, the positional output is dependent on pressure, finger size and environmental factors. For example, if you have a USB
+cable connected to the USB-C port and put it in your pants pocket without connecting it to anything, your finger will result in a
+different excitation than another person's finger who touches a different petal. This is not a super pratical scenario, but people
+have observed effects like this if flow3r has been on different surfaces (i.e. tables, couches). We tried our best to suppress
+these side effects in the ``.pressed`` and ``.pos`` outputs, but for example ``.raw_cap`` is heavily affected by it and there's
+little we can do about it.
+
+A more pratical side effect is that if you release a petal, the positional output will momentarily drift. This is bad for swipe
+gesture recognition, as this can easily be misread as a swipe. You might think that the ``.raw_cap`` channel may help suppressing
+this, but since ``.raw_cap`` also changes a lot during motion without liftoff, a trivial algorithm would suppress valid swipes.
+The current implementation of the ``Scroller`` widget does not use ``.raw_cap`` at all since any math we could come up with reasonable
+effort was situational at best, but typically detrimental to the feel.
+
+These liftoff artifacts (or lifton, for the counterpart at the beginning of a gesture) are a nuisance to many widgets in different
+forms. In general, we found it to be the be the best approach to ignore 20ms of the positional data from the beginning and/or end of
+each touch, depending on the use case. This pattern was found to be so common that the ``PetalLog`` class has been in great parts
+designed around facilitating the implementation of such rejection algorithms.
+
+Some users may instinctively use slow liftoff in order to make sure they don't accidentially introduce a motion, erroneously
+attributing these artifacts to their own performance rather than a shortcoming of the hardware. This is unfortunate, as these
+slow liftoffs are much harder to detect (we did some testing with ``.raw_cap`` but found no universally applicable pattern, there
+often is a visible kink in the data but it often occurs later than the artifacts, so if you investigate options like this make
+sure to exclude "red herrings" - we wasted a good few hours that could've been prevented by plotting *all* the data).
+
+The hardware is out there in the world, the best we can do at this point is to accept its performance, explain it to the user and
+then be **consistent** - if fast liftoffs are the most consistent way to work around these issues, we should go for them, even if
+for some they may be counterintuitive.
+
+**Data rates**
+
+As a rule of thumb, all (even) top petals are hooked up to one chip, all (odd) bottom petals to another, except for petal 2,
+which is connected to the "bottom" chip. This means for example that if you disable all bottom petals, petal 2 receives data
+much faster than the other top petals.
+
+Generally, each data channel that you collect (their amount being the integer value of ``.mode`` for each petal) takes about 0.75ms,
+however due to the asynchronous peripheral protocol we typically run a bit slower than that, expect the full cycle to take 2-3ms on
+top. Higher priority tasks (audio rendering, WiFi) may make this worse. Also, if the bottom chip is fully utilized (13 datapoints, 2
+from each bottom petal, 3 from petal 2) there is an additonal penalty resulting in a spin time of about 14ms.
+
+The ``PetalLog`` class (especially the ``index_offset_ms()`` method) is specifically designed to help dealing with those different
+data rates. Making widgets that feel the same-ish with different driver configurations is difficult: We're trying hard to make the
+provided widget library perform satisfactory at all configurations, but it is a time consuming task. Of course, if you write an
+application you only need to consider the driver configuration(s) that you actually are using. It is still a good idea to ask yourself
+whether some or the other data processing is supposed to occur in the time domain (for example, detecting motion in the last 100ms) or
+the index domain (for example, rejecting noise by averaging 4 samples).
+
+There is one caveat: If you do "hardcode" the behavior of a widget to a specific driver configuration, you should take care to set up the
+driver configuration so that all petals which use that widget actually run at the same expected data rate (i.e., same amount of active
+channels of that chip). Most commonly this affects petal 2 due to its irregular connection. For example, the Violin application, which
+extracts rubbing motion from all top petals, activates bottom petal channels that it does not use in order to make sure that petal 2 runs
+at the same data rate as the other top petals. Feel free to not use the widget auto-configuration at all and create your own for the purpose
+manually, or modify the autogenerated one after it has been created. It is meant as a mere helper, you may find reasons to ignore or enhance
+it at times.
+
+You might feel tempted to dynamically switch configuration, for example to run petals as buttons very fast and only enable positional
+output once they are touched. This would be great in theory and make many applications a bit snappier, however the chips exhibit strange
+undocumented glitches when configurations are changed in certain ways. Our approach to configuration changes at this point is to try
+to guarantee the validity of all datasets that you receive, but since these glitches often rare and difficult to track down we are
+overshooting and throwing away more data than needed. Changing configuration at this point in time results in 3 datasets being thrown
+away, resulting in a significant (~50ms typ.) gap in the data logs. We may be able to improve on this in some specific transition types,
+(i.e., channel number remains constant), but it is unclear if we will ever find the effort to implement this justifiable.
+
+
+**Miscellaneous quirks**
+
+- The top petals have quite a large deadzone near the outer tip.
+
+- The top petals like to "zig zag" around the center. For 1D value input the bottom petals are plain better.
+
+- The bottom petals are less noisy. To compensate, the top petals use stronger filtering in the non-logged positional output,
+  making them a bit slower.
+
+- Faster spin times do not only affect the log but also the built-in filters on the non-logged outputs, making especially the top
+  petals much more responsive. 
+
+- ``.raw_cap`` is not monotonic with respect to how much of the petal you cover: In fact, if you cover an entire top petal with multiple
+  flat fingers, it fairly consistently outputs lower values compared to the flat of the thumb. The causes for this behavior are unknown.
+
+
+Annex 1: Basics of complex numbers
+----------------------------------
+
+You may have noticed that the positional output is complex-valued. We find that it enables very concise 2D operations,
+but not everyone is familiar with them. If you don't wanna deal with it at all and use traditional coordinates instead,
+you can simply convert it to an x-y tuple like so:
+
+.. code-block:: python
+
+    # create complex number with real part 1 and imaginary part 3
+    pos = complex(1, 3)
+    # transform it into a x-y tuple
+    tuple_pos = (pos.real, pos.imag)
+
+If you do want to use them directly however, here's some basics:
+
+Typically we think of complex numbers as vector-like objects with two common representations: Above, we expressed them by their real
+and imaginary component, similar how traditional coordinates would use x and y components. Alternatively, we can express them as an
+angle and a length, as shown in the graphic below. Much of the magic of complex coordinate systems lies in the ability to seamlessly
+jump between those two representations.
+
+.. figure:: assets/Complex_number_illustration_modarg.svg
+    :scale: 150%
+    :target: https://commons.wikimedia.org/wiki/File:Complex_number_illustration_modarg.svg)
+    :alt: Illustration of an imaginary number on a cartesian plane with an arrow from origin to the number. The length of the arrow
+          and the angle between x axis and arrow are marked.
+    
+    by Kan8eDie / `CC BY-SA 3.0 Unported <https://creativecommons.org/licenses/by-sa/3.0/deed.en>`_
+
+Above, we have created a complex number by specifying the real and imaginary component. Let's create one in the "circular" representation
+instead and convert back and forth a little:
+
+.. code-block:: python
+
+    import cmath
+
+    # create number with angle of 45 degrees (math.tau / 8 in radians) and length of 2:
+    pos = cmath.rect(2, math.tau / 8)
+    
+    # as before, we can look at the x-y representation via the real and imaginary attributes:
+    pos_x = pos.real
+    pos_y = pos.imag
+
+    # we can look at the angular representation with standard library functions:
+    # get length, in this case 2
+    length = abs(pos)
+    # get angle, in this case math.tau / 8
+    angle = cmath.phase(pos)
+
+Let's manipulate those numbers a little. For starters, let's look at translation and scaling. This is fairly straigtforward and doesn't rely
+on the "angular" representation at all:
+
+.. code-block:: python
+
+    # make another number
+    offset = complex(2, 4)
+    # alternative notation: make a real number imaginary by appending the complex unit "j":
+    offset = 2 + 4j
+
+    # translate by 2 in the real direction and 4 in the imaginary direction
+    pos += offset
+
+    # scale both real and imaginary part by 2
+    pos *= 2
+
+This is not very exciting, so let's look at a cooler trick: Multiplication of two complex numbers adds their respective angles together,
+which can be used for rotation. Of course, this can be used together with scaling in the same operation; the scaling factor is simply the
+length of the complex number.
+
+.. code-block:: python
+
+    # create number with angle of 30 degrees (=360 / 12) and length of 0.1:
+    rotator = cmath.rect(0.1, math.tau / 12)
+
+    # save angle for future reference
+    prev_pos_angle = cmath.phase(pos)
+    prev_pos_length = abs(pos)
+
+    # apply the rotation and scaling
+    pos *= rotator
+
+    # check how much angle has changed: (angle_change % math.tau) equals* math.tau / 12
+    angle_change = cmath.phase(pos) - prev_pos_angle
+
+    # check how much angle has changed: length_change equals* 0.1:
+    length_change = abs(pos)/prev_pos_length
+
+    # *: plus minus floating point rounding errors
+
+Division works as with reals in that it undoes multiplication: It scales by the inverse (1/length), and rotates by the same angle
+but in the other direction. Of course, as with reals, multiplying by 0 destroys information so that dividing by 0 is impossible.
+
+.. code-block:: python
+
+    # complex numbers are nontruthy if both real and imaginary part are 0, else truthy
+    if rotator:
+        # we can undo rotation and scaling by dividing:
+        pos /= rotator
+        # this operation is slower than multiplication, but we can cache
+        # the inverse to make applying it fast:
+        antirotator = 1/rotator
+        # pos remains unchanged plus minus floating point rounding errors:
+        pos = (pos * rotator) * antirotator
+
+
+For rotating around a point other than the origin, simply translate and de-translate before and after the rotation:
+
+.. code-block:: python
+
+    pos -= offset
+    pos *= rotator
+    pos += offset
+
+As a practical example, here's how to set a bright RGB color from a petal position:
+
+We use a HSV representation because it is similarily circular, with hue being an angle and saturation being the distance
+from the (white) center. Notably, when saturation is 0, the value of hue doesn't matter. The final parameter, value,
+is fixed at "1" so we always get a bright color.
+
+*Note: A* ``Slider`` *widget would do a better job at preventing artifacts but let's keep things simple.*
+
+.. code-block:: python
+
+    # (in app.think())
+    petal = ins.captouch.petals[0]
+    if petal.pos is not None:
+        # angle of the position vector corresponds to hue
+        hue = cmath.phase(petal.pos)
+
+        # length of the position vector corresponds to saturation
+        sat = abs(petal.pos)
+        # length can be greater than 1 so we need to limit it 
+        # (but it is guaranteed to be able to reach 1 at any angle)
+        sat = max(sat, 1)
+
+        # max brightness always
+        val = 1
+
+        # transform to RGB because the LED driver uses that
+        rgb_col = st3m.ui.colours.hsv_to_rgb(hue, sat, val)
+        # apply to all LEDs
+        leds.set_all_rgb(*rgb_col)
+        leds.update()
+
+Annex 2: REPL workaround
+------------------------
+
+We've cornered ourselves there a little: some useful features of the captouch driver are synchronized to the ``think()`` cycle, but
+many early applications don't use the ``CaptouchState`` provided but instead create their own via ``captouch.read()`` (legacy, don't do
+this in new applications please). If this function were to trigger a reset in the ``.pressed`` attribute, half of the data would be
+thrown away and it would be easy to miss button presses. ``.log`` would drop frames in similar manner. You could do some sort of lazy
+evaluation of ``think()``'s object but that might just result in more subtle bugs if users aren't careful, we'd rather break loudly :D.
+Instead, the OS uses a special trigger function at the beginning of each ``think()``. To construct a proper ``CaptouchState`` in the
+repl we must call this function manually. Don't ever do it in applications tho, really.
+
+.. code-block:: python
+
+    import sys_captouch # in REPL only, never in applications!
+    sys_captouch.refresh_events() # your app will break in subtle and annoying ways
+    captouch_state = sys_captouch.read()
+
+
+Annex 3: Legacy API
+-------------------
+
+.. py:class:: CaptouchPetalState
+    :noindex:
+
+    .. py:attribute:: position
+        :type: Tuple(int, int)
+        
+        Similar to ``.raw_pos``, but not normalized. First element corresponds to real part, second element to imaginary.
+        For top petals about ``35000 * .raw_pos``, for bottom petals about ``25000 * .raw_pos + 5000`` (note that addition only
+        affects the real part, the imaginary part is always 0 for bottom petals).
+
+    .. py:attribute:: pressure
+        :type: int
+        
+        Similar to ``.raw_cap``, but not normalized. Depending on firmware version roughly about ``8000 * .raw_cap``, but may
+        or may not be always 0 if the petal is not pressed.
+
+
+.. py:function:: read() -> CaptouchState
+
+    Reads current captouch state from hardware and returns a snapshot in time.
+    ``.pressed`` and ``.log`` attributes are broken in the REPL.
+
+    Typically you'd want to use the captouch data provided by ``think()``, so this method for application purposes
+    is replaced with nothing. See workaround for reasoning.
+
+    What if you do *need* the captouch state outside of ``think()`` though? Well, chances are you don't, it just appears convenient:
+    We've seen this pattern a few times where ``think()`` requires a previous state, and the first such previous state is generated by
+    ``__init__()``, but this is an anti-pattern. Instead, set the previous state to ``None`` in ``on_enter()`` and handle that case
+    in ``think()``. The common consequence of doing otherwise is that after exiting and reentering an application the previous state
+    is very stale, which can lead to unintended behavior. Dedicated "first think" functionality really is the way to go in these cases.
+
+    Some example applications that ship with flow3r unfortunately use this pattern, and we should really clean that up, but we
+    didn't have time for this release yet. Apologies, IOU, will totally get around to it soon.

docs/app/api/colours.rst

+.. py:module:: colours
+
+``st3m.ui.colours`` module
+==========================
+
+Colour data is expressed in floats. ``r``, ``g``, ``b``, ``v``, ``s`` range from [0..1] and clamp beyond, ``h`` ranges from [0..math.tau] and overflows gracefully for values within [-100..100].
+
+Example (note the asterisk expanding the returned tuple):
+
+.. code-block:: python
+
+    import leds
+    import math
+    from st3m.ui import colours
+    leds.set_all_rgba(*colours.hsv_to_rgb(math.tau*5/6, 1, 1), 0.5)
+    leds.update()
+
+.. py:function:: hsv_to_rgb(h : float, s : float, v : float) -> Tuple[r : float, g: float, b: float]
+
+    Returns RGB tuple corresponding to the HSV input parameters.
+
+.. py:function:: rgb_to_hsv(r : float, g : float, b : float) -> Tuple[h : float, s: float, v: float]
+
+    Returns HSV tuple corresponding to the RGB input parameters.
+
+This module also provides some color constants:
+
+.. py:data:: BLACK
+.. py:data:: RED
+.. py:data:: GREEN
+.. py:data:: BLUE
+.. py:data:: WHITE
+.. py:data:: GREY
+.. py:data:: GO_GREEN
+.. py:data:: PUSH_RED

docs/api/ctx.rst → docs/app/api/ctx.rst

+.. _ctx API:
+
 ``ctx`` module
 ==============
 
+.. note::
+
+   Some functions might be better documented in the upstream uctx docs
+
+   https://ctx.graphics/uctx
+
+   Apologies for the divergence.
+
 .. automodule:: ctx
    :members:
    :undoc-members:

docs/app/api/input.rst

+.. py:module:: st3m.input
+
+``st3m.input`` module
+=====================
+
+All user facing classes contained herein are provided by the ``st3m.application.Application`` class, there is not really any
+need to import this module ever, and this documentation does not cover how to use these classes in an isolated way.
+
+``InputState``
+--------------
+
+This class and its contained classes should not be instantiated directly in an application, but instead the ``ins`` argument
+provided by ``.think(ins, delta_ms)`` method of the ``Application`` instance should be used.
+
+.. py:class:: InputState
+
+    .. py:attribute:: captouch
+       :type: captouch.CaptouchState
+
+        The state of the captouch surface, see documentation of the ``captouch`` module.
+
+    .. py:attribute:: buttons
+       :type: InputButtonState
+
+        The state of the shoulder buttons.
+
+    .. py:attribute:: imu
+       :type: IMUState
+
+        The state of the inertial measurement unit.
+
+    .. py:attribute:: temperature
+       :type: float
+
+        The internal ambient temperature in degree Celsius.
+
+    .. py:attribute:: battery_voltage
+       :type: float
+
+        The battery voltage in Volts.
+
+    .. py:attribute:: pressure
+       :type: float
+
+        The ambient pressure in Pascal. This is raw unfiltered data and very jittery. You might want to
+        apply some sort of filtering for most typical applications.
+
+
+.. py:class:: InputButtonState
+
+    .. py:attribute:: NOT_PRESSED
+       :type: int
+       :value: 0
+
+    .. py:attribute:: PRESSED_LEFT
+       :type: int
+       :value: -1
+
+    .. py:attribute:: PRESSED_RIGHT
+       :type: int
+       :value: 1
+
+    .. py:attribute:: PRESSED_DOWN
+       :type: int
+       :value: 2
+
+    .. py:attribute:: app
+       :type: int
+
+        State of the app shoulder button. May be any \*PRESSED\* constant.
+
+    .. py:attribute:: os
+       :type: int
+
+        State of the OS shoulder button. May be any \*PRESSED\* constant.
+
+        You probably don't wanna use this without engaging ``st3m.application.Application.override_os_button_back``
+        and/or ``st3m.application.Application.override_os_button_volume`` to avoid clashes with the operating
+        system's use of this button. Please check out the recommended override restrictions in their respective
+        documentations.
+
+    .. py:attribute:: app_is_left
+       :type: bool
+
+        Indicates whether the app button is the left shoulder button.
+
+        For context: While it is possible to operate flow3r by holding it in both hands not unlike a game controller,
+        this restricts access to the captouch surface. For many instruments, more speed and flexibility is needed,
+        so it is recommended to hold flow3r in one hand and play the captouch surface with the other. In this
+        configuration, one shoulder button is easily reachable by the holding hand's index finger while the other
+        is far away and can only be operated by the floating hand.
+
+        Since this results in a completely different user experience depending on the handedness of the user (being able
+        to control OS functionality with the holding hand vs. being able to control app functionality with the
+        holding hand) we decided to leave it up to the user to flip them in the global settings.
+
+        Since this also affects where hints might be placed on the display etc., making this feature hidden from application
+        developers would result in a second-class experience for one of the user groups. In order to prevent a default, we use
+        the abstract ``app`` and ``os`` terms instead of ``left`` and ``right``.
+
+
+.. py:class:: IMUState
+
+    .. py:attribute:: acc
+       :type: Tuple[float, float, float]
+
+        Acceleration in m/s^2. Includes gravity. See image below for mapping indices to axis.
+
+    .. py:attribute:: gyro
+       :type: Tuple[float, float, float]
+
+        Angular velocity in deg/s. See image below for mapping indices to axis.
+
+    .. py:attribute:: pressure
+
+        Duplicate of ``InputState.pressure``
+
+The axis of the IMU are arranged as follows, with the x-axis pointing from the center of the badge to the USB-C port and the z-axis
+pointing upwards through the display. Both ``acc`` and ``gyro`` tuples are arranged as ``(x,y,z)``. Image taken from the BMI270 datasheet.
+
+.. image:: assets/imu_axis.png
+
+``InputController``
+-------------------
+
+The ``InputController`` class holds edges of all button-like inputs, i.e. the shoulder buttons and captouch petals. Similar to
+``InputState`` it is not typically initialized by the user but rather provided in the ``.input`` attribute of the ``Application`` instance.
+The structure of these classes are not dissimilar, but there are some important differences to account for:
+
+.. code-block:: python
+
+    class App(st3m.application.Application):
+        def think(self, ins, delta_ms):
+            # mandatory for updating self.input
+            super().think(self, ins, delta_ms)
+
+            # True if the petal is currently being pressed or not
+            petal_pressed = ins.captouch.petal[0].pressed
+            # True if the petal has just switched from not pressed to pressed
+            rising_edge_of_petal_pressed = self.input.captouch.petal[0].whole.pressed
+
+            # True if the shoulder button is pressed down
+            button_pressed = ins.buttons.app == ins.buttons.DOWN
+            # True if the shoulder button has just switched from not pressed to pressed
+            rising_edge_of_button_pressed = self.input.buttons.app.middle.pressed
+
+All end nodes of ``InputController`` are ``Pressable`` objects:
+
+.. py:class:: Pressable
+
+    .. py:attribute:: pressed
+        :type: bool
+
+        True if an object hadn't been pressed in the cycle before but now is.
+
+        Note that this attribute follows a different naming convention than ``captouch.CaptouchPetalState.pressed``, which corresponds
+        more to the ``.down`` attribute (but not quite). For the purposes of documentation this name is an exception to the general
+        rule that "pressed" means "currently being touched", leading to somewhat silly sentences as in ``.down``. Sorry about that,
+        fixing this naming inconsistency is somewhat difficult.
+
+    .. py:attribute:: repeated
+        :type: bool
+        
+        True if an object has been pressed for sufficiently long to trigger a key repeat, see ``.repeat_enable()``.
+
+    .. py:attribute:: released
+        :type: bool
+
+        True if an object had been pressed in the cycle before but now isn't.
+
+        Note: ``.pressed`` and ``.released`` do not need to come in pairs. When entering an application, any Pressable will check
+        if it is pressed and assume that this interaction was used to enter the application, therefore pretending that it isn't
+        pressed right now. If the button was held the last time the application ran, that release event it will be supressed.
+        Equally, exiting an application while the button is held does not generate a release event, so any process that starts on
+        ``.pressed`` and ends on ``.released`` might not be terminated when closing an application.
+
+    .. py:attribute:: down
+        :type: bool
+
+        True if an object is being pressed and ``.pressed`` is ``False``.
+
+    .. py:attribute:: up
+        :type: bool
+
+        True if an object is not being pressed and ``.released`` is ``False``.
+
+    .. py:method:: repeat_enable(first : int = 400, subsequent : int = 200) -> None:
+
+        Enable key repeat functionality. Arguments are amount to wait in ms
+        until first repeat is emitted and until subsequent repeats are emitted.
+
+        Repeat is enabled by default on Pressables.
+
+    .. py:method:: repeat_disable(self) -> None:
+
+        Turns off key repeat functionality.
+
+The basic idea of ``Pressable`` is that all state attributes (i.e., ``.pressed``, ``.repeated``, ``.released``, ``.down``, ``.up``) are
+mutually exclusive. A single exception to that rule is that ``.down`` and ``.repeated`` may be true at the same time since repeats
+are enabled by default which causes flickering in ``.down`` which may be unexpected by many users.
+
+The full structure of ``InputController`` is as follows:
+
+.. py:class:: InputController
+
+    .. py:attribute:: captouch.petals
+        :type: list(PetalState)
+
+        A container for the edges of the captouch surface.
+
+    .. py:attribute:: buttons.app
+        :type: TriSwitchState
+
+        A container for the edges of the app button.
+
+    .. py:attribute:: buttons.os
+        :type: TriSwitchState
+
+        A container for the edges of the OS button.
+
+        You probably don't wanna use this without engaging ``st3m.application.Application.override_os_button_back``
+        and/or ``st3m.application.Application.override_os_button_volume`` to avoid clashes with the operating
+        system's use of this button. Please check out the recommended override restrictions in their respective
+        documentations.
+
+.. py:class:: TriSwitchState
+
+    .. py:attribute:: left
+        :type: Pressable
+
+    .. py:attribute:: middle
+        :type: Pressable
+    
+    .. py:attribute:: right
+        :type: Pressable
+
+.. py:class:: PetalState
+
+    .. py:attribute:: whole
+        :type: Pressable
+

docs/app/api/led_patterns.rst

+.. py:module:: led_patterns
+
+``st3m.ui.led_patterns`` module
+===============================
+
+None of these functions call ``leds.update()``, to actually see the changes you have to do that yourself!
+
+.. py:function:: highlight_petal_rgb(num : int, r : float, g : float, b : float, num_leds : int = 5) -> None
+
+    Sets the LED closest to the petal and num_leds-1 around it to the given rgb color.
+    If num_leds is uneven the appearance will be symmetric.
+
+.. py:function:: shift_all_hsv(h : float  = 0, s : float = 0, v : float = 0) -> None
+
+    Shifts all LEDs by the given values. Clips effective ``s`` and ``v``.
+
+.. py:function:: pretty_pattern() -> None
+
+    Generates a random pretty pattern and loads it into the LED buffer.
+
+.. py:function:: set_menu_colors() -> None
+
+    If not disabled in settings: Tries to load LED colors from /flash/menu_leds.json. Else, or in
+    case of missing file, call ``pretty_pattern()``. Note: There is no caching, it tries to attempt
+    to read the file every time, if you need something faster use ``leds.get_rgb`` to cache it in the
+    application.

docs/app/api/leds.rst

+``leds`` module
+===============
+
+The flow3r badge features a background task that helps creating smooth LED transitions. Users who choose
+to not expose themselves to blinking lights may do so by reducing the maximum slew rate in the system menu.
+This is the slew rate applications will default to on_enter(). There are rare occassions in which it is necessary
+for an application to function properly to increase this number, a good example is LED Painter which becomes
+unusable at too low of a slew rate.
+
+Please consider carefully if you wish to go above the user setting.
+
+.. code-block:: python
+
+    import leds
+    import math
+    from st3m.ui import colours
+
+    def on_enter(self, vm):
+        # mellow slew rate but never greater than user value
+        leds.set_slew_rate(min(leds.get_slew_rate(), 180))
+        leds.set_all_rgba(*colours.hsv_to_rgb(math.tau/3, 1, 1), 0.5)
+        leds.update()
+
+    def on_enter(self, vm):
+        # for response time critical applications like LED Painter: set practical minimum
+        leds.set_slew_rate(max(leds.get_slew_rate(), 200))
+        leds.set_all_rgba(*colours.hsv_to_rgb(math.tau/3, 1, 1), 0.5)
+        leds.update()
+
+
+.. automodule:: leds
+   :members:
+   :undoc-members:
+   :member-order: bysource

docs/app/api/view.rst

+``st3m.ui.view`` module
+=======================
+
+.. warning::
+
+    In earlier versions of flow3r there was no clear distinction between ``View`` and
+    ``st3m.ui.Application``. This was inconvenient so we are slowly transitioning to a new model
+    where an application may use views while being a distinct entity. This transition is incomplete
+    as we tried to wrap up the release for Chaos Communication Congress 2024 and many new features
+    rely on the new model.
+
+    This documentation is the old ``view`` documentation. We did not rewrite it for this release
+    yet as it is unclear how to use it future safely. Features like ``.on_enter()`` clash with
+    their ``st3m.ui.application`` counterpart, which will be rectified soon by introducing new
+    methods to ``Application``. Right now it is difficult to use them for both purposes.
+
+    We will try to keep existing ``View``/``ViewManager`` applications running and will provide a
+    clean way for new applications to do so soon, but in the meantime please don't try to get hacky
+    with these APIs because cannot maintain every edge case.
+
+Managing multiple views
+^^^^^^^^^^^^^^^^^^^^^^^
+
+If you want to write a more advanced application you probably also want to display more than
+one screen (or view as we call them).
+With just the Responder class this can become a bit tricky as it never knows when it is visible and
+when it is not. It also doesn't directly allow you to launch a new screen.
+
+To help you with that you can use a :py:class:`View` instead. It can tell you when
+it becomes visible, when it is about to become inactive (invisible) and you can
+also use it to bring a new screen or widget into the foreground or remove it
+again from the screen.
+
+Example 1: Managing two views
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In this example we use a basic `View` to switch between to different screens using a button. One screen
+shows a red square, the other one a green square. You can of course put any kind of complex processing
+into the two different views. We make use of an `InputController` again to handle the button presses.
+
+
+.. code-block:: python
+
+    from st3m.input import InputController
+    from st3m.ui.view import View
+    import st3m.run
+
+    class SecondScreen(View):
+        def __init__(self) -> None:
+            self.input = InputController()
+            self._vm = None
+
+        def on_enter(self, vm: Optional[ViewManager]) -> None:
+            self._vm = vm
+
+            # Ignore the button which brought us here until it is released
+            self.input._ignore_pressed()
+
+        def draw(self, ctx: Context) -> None:
+            # Paint the background black
+            ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
+            # Green square
+            ctx.rgb(0, 1, 0).rectangle(-20, -20, 40, 40).fill()
+
+        def think(self, ins: InputState, delta_ms: int) -> None:
+            self.input.think(ins, delta_ms) # let the input controller to its magic
+
+            # No need to handle returning back to Example on button press - the
+            # flow3r's ViewManager takes care of that automatically.
+
+
+    class Example(View):
+        def __init__(self) -> None:
+            self.input = InputController()
+            self._vm = None
+
+        def draw(self, ctx: Context) -> None:
+            # Paint the background black
+            ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
+            # Red square
+            ctx.rgb(1, 0, 0).rectangle(-20, -20, 40, 40).fill()
+
+
+        def on_enter(self, vm: Optional[ViewManager]) -> None:
+            self._vm = vm
+            self.input._ignore_pressed()
+
+        def think(self, ins: InputState, delta_ms: int) -> None:
+            self.input.think(ins, delta_ms) # let the input controller to its magic
+
+            if self.input.buttons.app.middle.pressed:
+                self._vm.push(SecondScreen())
+
+    st3m.run.run_view(Example())
+
+Try it using `mpremote`. The OS shoulder button (right shoulder unless swapped in settings) switches
+between the two views. To avoid that the still pressed button immediately closes `SecondScreen`
+we make us of a special method of the `InputController` which hides the pressed button from the
+view until it is released again.
+
+.. note::
+
+   Pressing the OS shoulder button in REPL mode will currently reset the badge.
+
+   Until this is fixed, you can test view switching by copying the app to your badge and
+   running from the menu.
+
+Example 2: Easier view management
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The above code is so universal that we provide a special view which takes care
+of this boilerplate: :py:class:`BaseView`. It integrated a local
+`InputController` on ``self.input`` and a copy of the :py:class:`ViewManager`
+which caused the View to enter on ``self.vm``.
+
+Here is our previous example rewritten to make use of `BaseView`:
+
+.. code-block:: python
+
+    from st3m.ui.view import BaseView
+    import st3m.run
+
+    class SecondScreen(BaseView):
+        def __init__(self) -> None:
+            # Remember to call super().__init__() if you implement your own
+            # constructor!
+            super().__init__()
+
+        def on_enter(self, vm: Optional[ViewManager]) -> None:
+            # Remember to call super().on_enter() if you implement your own
+            # on_enter!
+            super().on_enter(vm)
+
+        def draw(self, ctx: Context) -> None:
+            # Paint the background black
+            ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
+            # Green square
+            ctx.rgb(0, 1, 0).rectangle(-20, -20, 40, 40).fill()
+
+    class Example(BaseView):
+        def draw(self, ctx: Context) -> None:
+            # Paint the background black
+            ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
+            # Red square
+            ctx.rgb(1, 0, 0).rectangle(-20, -20, 40, 40).fill()
+
+        def think(self, ins: InputState, delta_ms: int) -> None:
+            super().think(ins, delta_ms) # Let BaseView do its thing
+
+            if self.input.buttons.app.middle.pressed:
+                self.vm.push(SecondScreen())
+
+    st3m.run.run_view(Example())
+
+
+In some cases, it's useful to know more about the view's lifecycle, so tasks
+like long-blocking loading screens can be synchronized with view transition
+animations. You'll also be interested in those events when doing unusual things
+with rendering, such as partial redraws or direct framebuffer manipulation.
+
+Here's the list of methods that you can implement in your class:
+
++-----------------------+-------------------------------------------------------------------------+
+| Function              | Meaning                                                                 |
++=======================+=========================================================================+
+| ``.on_enter(vm)``     | The view has became active and is about to start receiving ``.think()`` |
+|                       | and ``.draw()`` calls.                                                  |
++-----------------------+-------------------------------------------------------------------------+
+| ``.on_enter_done()``  | The view transition has finished animating; the whole screen is now     |
+|                       | under the control of the view.                                          |
+|                       |                                                                         |
+|                       | If you want to perform a long running blocking task (such as loading    |
+|                       | audio samples), it's a good idea to initiate it here to not block       |
+|                       | during the transition animation.                                        |
++-----------------------+-------------------------------------------------------------------------+
+| ``.on_exit()``        | The view has became inactive and started to transition out; no further  |
+|                       | ``.think()`` calls will be made until the next ``.on_enter()`` unless   |
+|                       | you return ``True`` from this handler.                                  |
+|                       |                                                                         |
+|                       | Do the clean-up of shared resources (such as LEDs or the media engine)  |
+|                       | here. This way you won't step onto the next view's shoes, as it may     |
+|                       | want to use them in its ``.on_enter()`` already.                        |
+|                       |                                                                         |
+|                       | **Note**: When returning ``True``, make sure not to handle input events |
+|                       | not meant for this view in your ``.think()`` handler during transition  |
+|                       | animation (see ``.is_active()`` below).                                 |
++-----------------------+-------------------------------------------------------------------------+
+| ``.on_exit_done()``   | The view transition has finished animating; no further ``.draw()`` or   |
+|                       | ``.think()`` calls will be made until the next ``.on_enter()``.         |
+|                       |                                                                         |
+|                       | A good place to do the clean-up of resources you have exclusive control |
+|                       | over, such as :ref:`bl00mbox` channels.                                 |
++-----------------------+-------------------------------------------------------------------------+
+| ``.show_icons()``     | Return ``True`` from it to have indicator icons drawn on top of your    |
+|                       | view by the system, just like in the main menu. Defaults to ``False``.  |
++-----------------------+-------------------------------------------------------------------------+
+
+:py:class:`ViewManager` also provides some methods to make handling common
+cases easier:
+
++-----------------------+-------------------------------------------------------------------------+
+| Function / property   | Meaning                                                                 |
++=======================+=========================================================================+
+| ``.is_active(view)``  | Returns a bool indicating whether the passed view is currently          |
+|                       | active. The active view is the one that's expected to be in control     |
+|                       | of user's input and the view stack. A view becomes active at            |
+|                       | ``.on_enter()`` and stops being active at ``.on_exit()``.               |
++-----------------------+-------------------------------------------------------------------------+
+| ``.transitioning``    | Whether a transition animation is currently in progress.                |
++-----------------------+-------------------------------------------------------------------------+
+| ``.direction``        | Returns the direction in which the currently active view has became one:|
+|                       |                                                                         |
+|                       |  - ``ViewTransitionDirection.NONE`` if it has replaced another view;    |
+|                       |  - ``ViewTransitionDirection.FORWARD`` if it was pushed into the stack; |
+|                       |  - ``ViewTransitionDirection.BACKWARD`` if another view was popped.     |
++-----------------------+-------------------------------------------------------------------------+
+
+On top of that, :py:class:`BaseView` implements an additional helper method
+``.is_active()``, which is simply a bit less awkward way to call
+``self.vm.is_active(self)``.

docs/app/api/widgets.rst

+``st3m.ui.widgets`` module
+==========================
+
+Basic Usage
+-----------
+
+flow3r widgets are objects that take raw user input and transform it in some or the other useful data. While this term
+is often strongly associated with a visual output, we use it in a more generic sense: The output of a widget doesn't
+necessarily go to the screen, but may be routed wherever.
+
+.. py:class:: Widget
+
+    .. py:method:: think(ins: InputState, delta_ms: int) -> None
+
+        Takes input from the hardware drivers and updates the state of the widget accordingly.
+
+    .. py:method:: on_exit() -> None
+
+        Suspends the widget. Should be called when ``.think()`` will not be executed anymore for a while
+        to make sure no stale data is used when the widget is reactivated later. This also allows the widget to
+        deallocate resources it might be using. Always call when application is exited.
+
+    .. py:method:: on_enter() -> None
+
+        Prepares/unsuspends the widget for immediate impending use. This may allocate resources needed for data processing
+        etc.
+
+The typical pattern for using widgets can be very simple. Since in many cases all the methods above are applied to all widgets at
+the same time, it is convenient to keep a list of all widgets, but for easily readable attribute access it makes also sense to keep
+a seperate named reference for each widget like so:
+
+.. code-block:: python
+
+    from st3m.ui import widgets
+
+    class App(Application):
+        def __init__(self, app_ctx):
+            super().__init__(app_ctx)
+            # keep direct references around for easy access...
+            self.some_widget = widgets.SomeWidget()
+            self.other_widget = widgets.OtherWidget()
+            # ...but also put them in a list for batch processing
+            self.widgets = [self.some_widget, self.other_widget]
+
+        def on_enter(self, vm):
+            super().on_enter(vm)
+            for widget in self.widgets:
+                widget.on_enter()
+
+        def on_exit(self):
+            super().on_exit()
+            for widget in self.widgets:
+                widget.on_exit()
+
+        def think(self, ins, delta_ms):
+            super().think(ins, delta_ms)
+            for widget in self.widgets:
+                widget.think(ins, delta_ms)
+
+            some_value = self.some_widget.attribute
+
+If you forget to call ``on_exit()``/``on_enter()`` you might not notice during testing but get strange bugs when people exit and
+reenter your application - it's best to always keep a list of all your widgets around so that you don't forget these method calls,
+
+If you don't need all your widgets all the time and want to save CPU by not calling ``.think()`` on all of them, it is also advised
+to use ``on_exit()``/``on_enter()`` as appropriate to avoid unwanted side effects.
+
+The primary motivation of widgets is to provide **captouch interaction processing**, but these APIs are a bit more complicated,
+so we start with the simpler group of sensor widgets:
+
+
+Sensor Widgets
+--------------
+
+
+These widgets provide common mathematical transformations and filtering of sensor data. They are designed to easily enhance
+user interfaces with motion control and similar interactions.
+
+.. py:class:: Altimeter(Widget)
+
+    .. py:method:: __init__(ref_temperature: Optional(float) = 15, filter_stages: int = 4, filter_coeff: float = 0.7) -> None
+
+        Initializes a pressure-based altimeter. ``ref_temperature`` initializes the attribute of the same name. Since the
+        ``meters_above_sea`` output is very noisy a multistage pole filter is provided. The number of stages is set by
+        ``filter_stages`` and must be 0 or greater. ``filter_coeff`` initializes the attribute of the same name.
+
+    .. py:attribute:: meters_above_sea
+       :type: float
+
+        Rough estimate of altitude in meters. The "above sea" bit is mostly theoretical, see ``.temperature``, it is best used
+        as a relative measurement. Very sensitive to pretty much everything, like moving your arms, closing windows and doors,
+        wind, you name it. Heavy filtering is recommended, meaning for practical purposes you may expect a very slow response
+        time in the magnitude of >10s.
+
+    .. py:attribute:: ref_temperature
+       :type: Optional(float)
+
+        Set to None to use the built-in temperature sensor for calculations, else use the value supplied here - wait, why
+        wouldn't you use the built-in sensor? The reason for this that temperature is just a proxy of the density curve of the
+        air column, from your position all the way up into space! This total weight is the main source of local pressure, so
+        we can use it to estimate how much air column there is on top of you, i.e. how high up in the athmosphere you are. So
+        if you're in a heated room, your local temperature is misrepresentative of most of that column, and as soon as you step
+        outside (or in a differently heated room) your reading will jump all over the place. Using the integrated sensor therefore
+        doesn't really make the data better unless you are outside - it just makes it more jumpy.
+
+        Pressure between rooms is fairly consistent, albeit noisy. Say we don't care for the absolute height, but how height
+        changes over time as we move flow3r up and down: Using a constant temperature guesstimate at worst introduces a minor
+        scaling error but removes all jumps from local temperature changes. It is almost always the better option to ignore it.
+
+        If you do want to perform actual absolute altimetry we advise polling weather services for local temperature data. Also
+        barometric pressure has other sources of error that you might take into account. It's probably best if widgets don't
+        silently connect to the web to poll data, so this will never be a fully automatic feature.
+
+    .. py:attribute:: filter_coeff
+       :type: float
+
+        Cutoff coefficient for all filter stages. Should typically be between 0 and 1.  Each stages is updated once per ``.think()``
+        by the rule ``output = input * (1-filter_coeff) + output * filter_coeff``, so that higher ``filter_coeff`` values result in
+        slower response. Changing this parameter while the widget is running will not result in instant response as the change
+        needs to propagate through the filter first, if you need two different response behaviors at the same time it is best to
+        create two seperate instances or filter externally.
+
+        *Note: This filter is not synchronized to actual data refresh rate but rather to the think rate, meaning that the same
+        sample may occur in the buffer multiple times in succession or that data points might be skipped, and that response time
+        is a function of think rate. The actual data refresh rate is >20ms.*
+
+.. py:class:: Inclinometer(Widget)
+
+    .. py:method:: __init__(buffer_len: int = 2) -> None:
+
+        Initializes a inclinometer. Noise can be removed with a windowed averaging filter, ``buffer_len`` sets the maximum window
+        length of that filter. At runtime you may switch between filter lengths with the ``filter_len`` attribute as you wish, but
+        its maximum must be set at initialization. Must be greater than 0.
+
+    .. py:attribute:: inclination
+       :type: float
+
+        Polar coordinates: How much flow3r is tilted compared to lying face-up on a table, in radians. Facing up is represented
+        by a value of 0, facing down by π, sideways is π/2 et cetera.
+
+        Range: [0..π]
+
+    .. py:attribute:: azimuth
+       :type: float
+
+        Polar coordinates: Informally this parameter represents which petal is pointing up most: Say flow3r is upright
+        (``.inclination`` = π/2), a value of 0 means the USB-C port is pointing up. A positive value indicates that it is rotated
+        clockwise from there, as a steering wheel analogy you're "turning right". Of course this value works just the same for all
+        values of ``inclination``, but there is a caveat: If fully facing up or facing down, so that no petal is pointing up more than the
+        others, this parameter will jump around with the random noise it picks up. If this causes you issues it's best to check
+        ``inclination`` whether it is tilted enough for your purposes.
+
+        Range: [-π..π]
+
+    .. py:attribute:: roll
+       :type: float
+
+        Aviation coordinates: How far flow3r is tilted sideways to the right, in radians.
+
+        Range: [-π..π]
+
+        See coordinate system note at the ``pitch`` attribute description.
+
+    .. py:attribute:: pitch
+       :type: float
+
+        Aviation coordinates: How far flow3r is tilted backwards, in radians.
+
+        Range: [-π..π]
+
+        *Coordinate system note:* Isn't yaw missing here? Yes, but we can't obtain it from the gravity vector, so we're skipping it.
+        Also you might notice that normally the pitch angle only covers a range of π, this implementation however spans the full circle.
+        
+        This has a side effect: Normally, the pitch angle range is limited to π, but here we go full circle. We do this to avoid
+        a discontinuity that would normally jump between interpreting, say, a unrolled pitch-down to a half-rolled pitch-up. The price
+        we pay for this is that our modified coordinate system fails when facing down: With full range on both angles it is undecidable
+        whether these positions come from rolling or pitching by an angle of π. We distribute it based on ``azimuth`` instead, which
+        tends to be jumpy in that area, so both ``pitch`` and ``roll`` experience full-range random jumps in that zone.
+
+    .. py:attribute:: filter_len
+       :type: int
+
+        Sets the length of the windowed averaging filter. Copies the value ``buffer_len`` at initialization. Must be greater than
+        zero and smaller or equal to ``buffer_len``. This parameter only affects the values of the output attributes and doesn't
+        change any of the available data, so you may even change it several times within the same ``think`` cycle and retrieve the
+        same output attributes with different response times if needed.
+
+        *Note: This filter is not synchronized to actual data refresh rate but rather to the think rate, meaning that the same
+        sample may occur in the buffer multiple times in succession or that data points might be skipped, and that response time
+        is a function of think rate. The actual data refresh rate is >10ms.*
+
+
+Captouch Widgets
+----------------
+
+The raw output data of the captouch driver is quite rough around the edges as described in the ``captouch`` module documentation.
+The widgets provided here aim to provide an intermediate layer to remove all the pitfalls that this poses; you might think that
+many of these are trivial and barely deserving of their own classes, but the devil is in the details. They are not without quirks,
+but if we ever figure out how to fix them these updates will directly benefit your application, and there is value in having
+consistent quirks across the entire userland if possible.
+
+These widgets often don't work with the default captouch driver configuration. To make it easy to generate a configuration that
+works for all widgets used at a given time, each widget can add its requirements to a ``captouch.Config`` object. Typically this
+is done at initialization:
+
+.. code-block:: python
+
+    from st3m.ui import widgets
+    import captouch
+
+    class App(Application):
+        def __init__(self, app_ctx):
+            super().__init__(app_ctx)
+            # create a default configuration
+            self.captouch_config = captouch.Config.default()
+            # create some widgets and add requirements to the config above
+            self.slider = widgets.Slider(self.captouch_config, 4)
+            self.scroller = widgets.Scroller(self.captouch_config, 2)
+
+            self.widgets = [self.slider, self.scroller]
+
+        def on_enter(self, vm):
+            super().on_enter(vm)
+            # apply the configuration when entering
+            self.captouch_config.apply()
+            for widget in self.widgets:
+                widget.on_enter()
+
+        # (other methods same as in the general Widget example above)
+
+The example above is a bit wasteful: The ``.default()`` configuration activates a lot of data channels that we might not be using
+and which slow down the data rates of the channels that we actually care about. In the example above, the ``Scroller`` widget
+would benefit a lot from the 3x-ish increase in data rate that starting with an ``.empty()`` configuration would yield (see
+the ``captouch`` module documentation for details).
+ 
+However, not all captouch data access is happening via widgets; an application might use edges from ``st3m.input.InputController``
+or primitive position output from ``st3m.input.InputState``. When starting with a ``.default()`` configuration and only adding
+widget requirements these are guaranteed to work; however, for best performance it is ideal to start with an ``.empty()`` one
+and add them manually as we will demonstrate below.
+
+One more case would be that not all widgets are in use at the same time; in that case, it makes sense to create different configs
+to switch between. We can only pass one configuration to the initializer, but that just forwards it to the ``.add_to_config()``
+method, which we can call as many times with different config objects as we like. The initializer must still receive a valid
+config to serve as a beginner's trap safety net.
+
+Here's a slightly more advanced example:
+
+.. code-block:: python
+
+    from st3m.ui import widgets
+    import captouch
+
+    class App(Application):
+        def __init__(self, app_ctx):
+            super().__init__(app_ctx)
+            self.captouch_configs = [None, None]
+            for x in range(2):
+                # create a empty configuration
+                captouch_config = captouch.Config.empty()
+                # manually add petal 5 in button mode
+                captouch_config[5].mode = 1
+                self.captouch_configs[x] = captouch_config
+
+            # the slider widget is only sometimes in use:
+            self.slider_enabled = True
+
+            # add both widget requirements to configs[0]
+            self.slider = widgets.Slider(self.captouch_configs[0], 4)
+            self.scroller = widgets.Scroller(self.captouch_configs[0], 2)
+
+            # only add the scroller configs[1]:
+            self.scroller.add_to_config(self.captouch_configs[1])
+
+            self.widgets = [self.slider, self.scroller]
+
+        def on_enter(self, vm):
+            super().on_enter(vm)
+            # select config we want
+            config_index = 0 if self.slider_active else 1
+            self.captouch_configs[config_index].apply()
+            for widget in self.widgets:
+                if self.slider_enabled or widget != self.Slider:
+                    widget.on_enter()
+
+        def on_exit(self):
+            super().on_exit()
+            # calling widget.on_exit() multiple times without pairing it with on_enter()
+            # is okay so we don't need to check here
+            for widget in self.widgets:
+                widget.on_exit()
+
+        def think(self, ins, delta_ms):
+            super().think(ins, delta_ms)
+            for widget in self.widgets:
+                widget.think(ins, delta_ms)
+
+            if self.input.captouch.petals[5].whole.pressed:
+                self.slider_enabled = not self.slider_enabled
+                if self.slider_enabled:
+                    self.slider.on_enter()
+                else:
+                    self.slider.on_exit()
+                config_index = 0 if self.slider_active else 1
+                self.captouch_configs[config_index].apply()
+
+            if self.slider_enabled:
+                do_thing_with_slider_value(self.slider.pos)
+            do_thing_with_scroller_value(scroller_value = self.scroller.pos)
+
+With that out of the way, let's finally look at the base class of all captouch widgets:
+
+.. py:class:: CaptouchWidget(Widget)
+
+    .. py:method:: __init__(config: CaptouchConfig, gain: complex = 1, constraint: Constraint = None, \
+                            friction: float = 0.7, bounce: float = 1) -> None
+
+        Initializes widget and adds its requirements to ``config`` via ``.add_to_config()``.
+        This is mandatory as most widgets do not work with the default captouch driver configuration.
+        The other arguments initialize the parameters of the same names.
+
+    .. py:method:: add_to_config(config: CaptouchConfig) -> None
+
+        Adds the requirements of the widget to a captouch config object, see examples above.
+
+    .. py:attribute:: pos
+       :type: complex
+
+        Main data output of widget. Individual behavior is documented in the subclasses below. Default value is 0 unless specified
+        otherwise.
+
+        While this is primary intended for read access, writing is allowed. This is useful for example for initializing a Slider
+        to a default value or resetting a widget that does relative movements. Writing does not automatically apply the current
+        ``.constraint`` but you can do so manually as with the "bad bounce" workaround (see Constraints section for more details).
+
+    .. py:attribute:: active
+       :type: bool
+
+        Read only: Whether the widget is currently interacted with or not. Updated by ``.think()``.
+
+    .. py:attribute:: gain
+       :type: complex
+
+        Multiplier that scales how much a physical captouch interaction changes the ``.pos`` output. Since it is complex-valued
+        this can also include a rotation around the origin, for example when ``cmath.rect(scaling_factor, angle)`` or
+        ``scaling_factor * captouch.PETAL_ROTORS[petal_index]`` is used.
+
+    .. py:attribute:: constraint
+       :type: Constraint
+
+        Limits the possible values of ``.pos`` after ``.gain`` is applied. Note that swapping or modifying constraint after
+        widget initialization may result in un-"physical" side effects, see documentation of the ``Constraint`` class below.
+        Multiple widgets may use the same constraint, this results in the same behavior as individual identical constraints.
+
+    .. py:attribute:: friction
+       :type: float
+
+        How fast the speed of ``pos`` decays if it is moving freely. Must be positive or zero. Not used by all widgets.
+
+    .. py:attribute:: bounce
+       :type: float
+
+        By how much the absolute speed is multiplied when colliding with a constraint wall. Must be positive or zero.
+        Not used by all widgets.
+
+Single-Petal Widgets
+^^^^^^^^^^^^^^^^^^^^
+
+.. py:class:: PetalWidget(CaptouchWidget)
+
+    Parent class of a widget that reacts to a single petal.
+
+    .. py:method:: __init__(config: CaptouchConfig, petal: int, **kwargs) -> None
+
+        Initializes widget and ties it to the petal with the index specified by the `petal` argument. ``config`` and ``**kwargs``
+        are forwarded to the initializer of ``CaptouchWidget``.
+
+.. py:class:: Slider(PetalWidget)
+
+    This widget allows users to set an application parameter with absolute touch position. If no ``constraint`` argument
+    is passed to the initializer it defaults to a unit circle constraint. Ignores ``friction`` and ``bounce`` parameters.
+
+    .. py:attribute:: pos
+       :type: complex
+
+        Absolute position of the last touch.
+
+.. py:class:: Scroller(PetalWidget)
+
+    This widget allows users to change an application parameter with relative touch position. If the friction is set to 1,
+    this can be used as an "incremental" ``Slider``, friction values between 0 and 1 allow for crude swipe gestures. *Note:
+    due to captouch data limitations it is very hard to not swipe at least a little bit, expect some residual drift when
+    planning a UI.*
+
+    If no ``constraint`` argument is passed or it is `None` to the initializer it throws a ``TypeError`` since else ``pos``
+    might grow without bounds up into the ``NaN`` range which would cause harder-to-detect rounding issues and crashes.
+
+    .. py:attribute:: pos
+       :type: complex
+
+        Relative position that is changed incrementally with touch.
+
+Multi-Petal Widgets
+^^^^^^^^^^^^^^^^^^^
+
+.. py:class:: MultiSlider(CaptouchWidget)
+
+    If no ``constraint`` argument is passed to the initializer this defaults to a unit circle constraint. Ignores ``friction``
+    and ``bounce`` parameters.
+
+    .. py:attribute:: pos
+       :type: complex
+
+        Takes all top petals as one big surface. Active if only one touch is registered on that surface (i.e., if at most 2
+        adjacent petals are pressed). Normalized so that the absolute value is typically between 0.5 and 1. The center values
+        cannot be reached, but it is initialized to 0 anyways.
+
+Constraints
+^^^^^^^^^^^
+
+Constraints limit the ``pos`` output of a ``CaptouchWidget`` to a region. This needn't be simple value clipping but can also include
+more advanced behavior such as overflowing or bouncing off walls. Since you probably want different behavior depending on whether
+you're "holding" the "ball" or not, and the asynchronous nature of the petal log may result in multiple states per think, these needed
+to be integrated deeply into the widgets. They're on this weird state of complexity where it feels tempting to either go full game
+engine (which would however be hard to justify effort) or radically simplifying them (which would however restrict some use cases).
+
+We decided to keep them on this middle ground for now, which has some implications:
+
+- Resizing/centering/rotating them during runtime may result in "bad bounces" if a widget is at a position that was previously within
+  the constraint but now is outside. As a mitigation call ``.apply_hold()`` manually to avoid such a bounce.
+
+- Constraints can't give positional outputs a physical "shapeness" to collide with others asthey have no concept of mass or motion
+  of self.
+
+- There is no computationally efficient way to combine them to create a "game level". If you aim for this it is best to make one
+  "giant constraint" which implements all the math from scratch.
+
+.. py:class:: Constraint
+
+    .. py:method:: __init__(size: [float, complex] = 1, center: complex = 0j, rotation: float = 0)
+
+        Creates a new constraint located at the given center with given rotation. The parameters initialize the attributes
+        of the same name.
+
+        *Note: Not all constraint subclasses must accept these parameters, we merely grouped them up here since all
+        constraints we provide use them. If you implement your own application specific constraint feel free to use
+        arbitrary arguments for the initializer.* 
+
+    .. py:method:: apply_hold(pos: complex) -> complex
+
+        Returns ``pos`` limited to whatever range the constraint defines.
+        Typically called automatically by the widget(s) using the constraint, manual calls are rarely needed.
+
+    .. py:method:: apply_free(pos: complex, vel: complex, bounce: float) -> (complex, complex)
+
+        Returns ``(pos, vel)`` tuple. Like apply_hold, but allows for bouncing off of walls.
+        Typically called automatically by the widget(s) using the constraint, manual calls are rarely needed. Not all
+        widgets call this method; at this point in time, only the ones that use ``friction`` do so. If you inherit from
+        this base class and do not define it, it defaults to limiting ``pos`` with ``.apply_hold()`` and leaving ``vel``
+        untouched.
+
+    .. py:attribute:: size
+        :type: complex
+
+        Linearily scales the size of the constraint. The real component must be greater than 0, the imaginary component
+        must be no less than 0, else setting will raise a ValueError. Setting it with a float or a complex number whose
+        imaginary component is 0 results in a "square-shaped" value, or ``size = complex(size.real, size.real)``.
+    
+        *Note: Not all constraint subclasses must have this attribute, see* ``__init__`` *note. Changing this attribute at
+        runtime may result in "bad bounces", see above.*
+
+    .. py:attribute:: center
+        :type: complex
+
+        Shifts the center of the constraint.
+
+        *Note: Not all constraint subclasses must have this attribute, see* ``__init__`` *note. Changing this attribute at
+        runtime may result in "bad bounces", see above.*
+
+    .. py:attribute:: rotation
+        :type: float
+
+        Rotates the constraint in radians.
+
+        *Note: Not all constraint subclasses must have this attribute, see* ``__init__`` *note. Changing this attribute at
+        runtime may result in "bad bounces", see above.*
+
+.. py:class:: Rectangle(Constraint)
+
+    A rectangular constraint. Size corresponds to side length.
+
+.. py:class:: ModuloRectangle(Constraint)
+
+    Like ``Rectangle``, but values overflow, ``bounce`` is ignored.
+
+.. py:class:: Ellipse(Constraint)
+
+    An elliptic constraint. Size corresponds to radii.