Skip to content
Snippets Groups Projects
  1. Feb 12, 2019
    • Damien George's avatar
      py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API. · eee1e884
      Damien George authored
      These macros could in principle be (inline) functions so it makes sense to
      have them lower case, to match the other C API functions.
      
      The remaining macros that are upper case are:
      - MP_OBJ_TO_PTR, MP_OBJ_FROM_PTR
      - MP_OBJ_NEW_SMALL_INT, MP_OBJ_SMALL_INT_VALUE
      - MP_OBJ_NEW_QSTR, MP_OBJ_QSTR_VALUE
      - MP_OBJ_FUN_MAKE_SIG
      - MP_DECLARE_CONST_xxx
      - MP_DEFINE_CONST_xxx
      
      These must remain macros because they are used when defining const data (at
      least, MP_OBJ_NEW_SMALL_INT is so it makes sense to have
      MP_OBJ_SMALL_INT_VALUE also a macro).
      
      For those macros that have been made lower case, compatibility macros are
      provided for the old names so that users do not need to change their code
      immediately.
      eee1e884
  2. Sep 20, 2018
  3. Dec 29, 2017
  4. Dec 28, 2017
    • Damien George's avatar
    • Damien George's avatar
      py/parse: Compress rule pointer table to table of offsets. · 0016a453
      Damien George authored
      This is the sixth and final patch in a series of patches to the parser that
      aims to reduce code size by compressing the data corresponding to the rules
      of the grammar.
      
      Prior to this set of patches the rules were stored as rule_t structs with
      rule_id, act and arg members.  And then there was a big table of pointers
      which allowed to lookup the address of a rule_t struct given the id of that
      rule.
      
      The changes that have been made are:
      - Breaking up of the rule_t struct into individual components, with each
        component in a separate array.
      - Removal of the rule_id part of the struct because it's not needed.
      - Put all the rule arg data in a big array.
      - Change the table of pointers to rules to a table of offsets within the
        array of rule arg data.
      
      The last point is what is done in this patch here and brings about the
      biggest decreases in code size, because an array of pointers is now an
      array of bytes.
      
      Code size changes for the six patches combined is:
      
         bare-arm:  -644
      minimal x86: -1856
         unix x64: -5408
      unix nanbox: -2080
            stm32:  -720
          esp8266:  -812
           cc3200:  -712
      
      For the change in parser performance: it was measured on pyboard that these
      six patches combined gave an increase in script parse time of about 0.4%.
      This is due to the slightly more complicated way of looking up the data for
      a rule (since the 9th bit of the offset into the rule arg data table is
      calculated with an if statement).  This is an acceptable increase in parse
      time considering that parsing is only done once per script (if compiled on
      the target).
      0016a453
    • Damien George's avatar
    • Damien George's avatar
    • Damien George's avatar
      py/parse: Pass rule_id to push_result_rule, instead of passing rule_t*. · 815a8cd1
      Damien George authored
      Reduces code size by eliminating quite a few pointer dereferences.
      815a8cd1
    • Damien George's avatar
      py/parse: Break rule data into separate act and arg arrays. · 845511af
      Damien George authored
      Instead of each rule being stored in ROM as a struct with rule_id, act and
      arg, the act and arg parts are now in separate arrays and the rule_id part
      is removed because it's not needed.  This reduces code size, by roughly one
      byte per grammar rule, around 150 bytes.
      845511af
    • Damien George's avatar
      py/parse: Split out rule name from rule struct into separate array. · 1039c5e6
      Damien George authored
      The rule name is only used for debugging, and this patch makes things a bit
      cleaner by completely separating out the rule name from the rest of the
      rule data.
      1039c5e6
  5. Dec 11, 2017
  6. Nov 16, 2017
    • Damien George's avatar
      py/objstr: Make mp_obj_new_str_of_type check for existing interned qstr. · 1f1d5194
      Damien George authored
      The function mp_obj_new_str_of_type is a general str object constructor
      used in many places in the code to create either a str or bytes object.
      When creating a str it should first check if the string data already exists
      as an interned qstr, and if so then return the qstr object.  This patch
      makes the function have such behaviour, which helps to reduce heap usage by
      reusing existing interned data where possible.
      
      The old behaviour of mp_obj_new_str_of_type (which didn't check for
      existing interned data) is made available through the function
      mp_obj_new_str_copy, but should only be used in very special cases.
      
      One consequence of this patch is that the following expression is now True:
      
          'abc' is ' abc '.split()[0]
      1f1d5194
  7. Oct 04, 2017
    • Damien George's avatar
      all: Remove inclusion of internal py header files. · a3dc1b19
      Damien George authored
      Header files that are considered internal to the py core and should not
      normally be included directly are:
          py/nlr.h - internal nlr configuration and declarations
          py/bc0.h - contains bytecode macro definitions
          py/runtime0.h - contains basic runtime enums
      
      Instead, the top-level header files to include are one of:
          py/obj.h - includes runtime0.h and defines everything to use the
              mp_obj_t type
          py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
              and defines everything to use the general runtime support functions
      
      Additional, specific headers (eg py/objlist.h) can be included if needed.
      a3dc1b19
  8. Jul 31, 2017
  9. Feb 24, 2017
    • Damien George's avatar
      py/parse: Simplify handling of errors by raising them directly. · f615d82d
      Damien George authored
      The parser was originally written to work without raising any exceptions
      and instead return an error value to the caller.  But it's now required
      that a call to the parser be wrapped in an nlr handler, so we may as well
      make use of that fact and simplify the parser so that it doesn't need to
      keep track of any memory errors that it had.  The parser anyway explicitly
      raises an exception at the end if there was an error.
      
      This patch simplifies the parser by letting the underlying memory
      allocation functions raise an exception if they fail to allocate any
      memory.  And if there is an error parsing the "<id> = const(<val>)" pattern
      then that also raises an exception right away instead of trying to recover
      gracefully and then raise.
      f615d82d
    • Damien George's avatar
      py: Create str/bytes objects in the parser, not the compiler. · 5255255f
      Damien George authored
      Previous to this patch any non-interned str/bytes objects would create a
      special parse node that held a copy of the str/bytes data.  Then in the
      compiler this data would be turned into a str/bytes object.  This actually
      lead to 2 copies of the data, one in the parse node and one in the object.
      The parse node's copy of the data would be freed at the end of the compile
      stage but nevertheless it meant that the peak memory usage of the
      parse/compile stage was higher than it needed to be (by an amount equal to
      the number of bytes in all the non-interned str/bytes objects).
      
      This patch changes the behaviour so that str/bytes objects are created
      directly in the parser and the object stored in a const-object parse node
      (which already exists for bignum, float and complex const objects).  This
      reduces peak RAM usage of the parse/compile stage, simplifies the parser
      and compiler, and reduces code size by about 170 bytes on Thumb2 archs,
      and by about 300 bytes on Xtensa archs.
      5255255f
    • Damien George's avatar
      py/parse: Allow parser/compiler consts to be bignums. · 74f4d2c6
      Damien George authored
      This patch allows uPy consts to be bignums, eg:
      
          X = const(1 << 100)
      
      The infrastructure for consts to be a bignum (rather than restricted to
      small integers) has been in place for a while, ever since constant folding
      was upgraded to allow bignums.  It just required a small change (in this
      patch) to enable it.
      74f4d2c6
  10. Feb 16, 2017
    • Damien George's avatar
      py/grammar: Group no-compile grammar rules together to shrink tables. · 71019ae4
      Damien George authored
      Grammar rules have 2 variants: ones that are attached to a specific
      compile function which is called to compile that grammar node, and ones
      that don't have a compile function and are instead just inspected to see
      what form they take.
      
      In the compiler there is a table of all grammar rules, with each entry
      having a pointer to the associated compile function.  Those rules with no
      compile function have a null pointer.  There are 120 such rules, so that's
      120 words of essentially wasted code space.
      
      By grouping together the compile vs no-compile rules we can put all the
      no-compile rules at the end of the list of rules, and then we don't need
      to store the null pointers.  We just have a truncated table and it's
      guaranteed that when indexing this table we only index the first half,
      the half with populated pointers.
      
      This patch implements such a grouping by having a specific macro for the
      compile vs no-compile grammar rules (DEF_RULE vs DEF_RULE_NC).  It saves
      around 460 bytes of code on 32-bit archs.
      71019ae4
  11. Jan 17, 2017
  12. Nov 15, 2016
  13. Nov 02, 2016
    • Colin Hogben's avatar
      py: Fix wrong assumption that m_renew will not move if shrinking · f9b6b37c
      Colin Hogben authored
      In both parse.c and qstr.c, an internal chunking allocator tidies up
      by calling m_renew to shrink an allocated chunk to the size used, and
      assumes that the chunk will not move.  However, when MICROPY_ENABLE_GC
      is false, m_renew calls the system realloc, which does not guarantee
      this behaviour.  Environments where realloc may return a different
      pointer include:
      
      (1) mbed-os with MBED_HEAP_STATS_ENABLED (which adds a wrapper around
      malloc & friends; this is where I was hit by the bug);
      
      (2) valgrind on linux (how I diagnosed it).
      
      The fix is to call m_renew_maybe with allow_move=false.
      f9b6b37c
  14. Sep 23, 2016
  15. Jun 06, 2016
  16. May 20, 2016
  17. May 10, 2016
  18. Apr 14, 2016
    • Damien George's avatar
      py: Simplify "and" action within parser by making ident-rules explicit. · 0c1de1cd
      Damien George authored
      Most grammar rules can optimise to the identity if they only have a single
      argument, saving a lot of RAM building the parse tree.  Previous to this
      patch, whether a given grammar rule could be optimised was defined (mostly
      implicitly) by a complicated set of logic rules.  With this patch the
      definition is always specified explicitly by using "and_ident" in the rule
      definition in the grammar.  This simplifies the logic of the parser,
      making it a bit smaller and faster.  RAM usage in unaffected.
      0c1de1cd
  19. Apr 13, 2016
  20. Mar 19, 2016
  21. Feb 23, 2016
  22. Jan 12, 2016
  23. Jan 08, 2016
  24. Jan 07, 2016
    • Damien George's avatar
      py/parse: Improve constant folding to operate on small and big ints. · 22b22650
      Damien George authored
      Constant folding in the parser can now operate on big ints, whatever
      their representation.  This is now possible because the parser can create
      parse nodes holding arbitrary objects.  For the case of small ints the
      folding is still efficient in RAM because the folded small int is stored
      inplace in the parse node.
      
      Adds 48 bytes to code size on Thumb2 architecture.  Helps reduce heap
      usage because more constants can be computed at compile time, leading to
      a smaller parse tree, and most importantly means that the constants don't
      have to be computed at runtime (perhaps more than once).  Parser will now
      be a little slower when folding due to calls to runtime to do the
      arithmetic.
      22b22650
    • Damien George's avatar
      py/parse: Optimise away parse node that's just parenthesis around expr. · 93b37262
      Damien George authored
      Before this patch, (x+y)*z would be parsed to a tree that contained a
      redundant identity parse node corresponding to the parenthesis.  With
      this patch such nodes are optimised away, which reduces memory
      requirements for expressions with parenthesis, and simplifies the
      compiler because it doesn't need to handle this identity case.
      
      A parenthesis parse node is still needed for tuples.
      93b37262
  25. Dec 18, 2015
    • Damien George's avatar
      py: Add MICROPY_ENABLE_COMPILER and MICROPY_PY_BUILTINS_EVAL_EXEC opts. · dd5353a4
      Damien George authored
      MICROPY_ENABLE_COMPILER can be used to enable/disable the entire compiler,
      which is useful when only loading of pre-compiled bytecode is supported.
      It is enabled by default.
      
      MICROPY_PY_BUILTINS_EVAL_EXEC controls support of eval and exec builtin
      functions.  By default they are only included if MICROPY_ENABLE_COMPILER
      is enabled.
      
      Disabling both options saves about 40k of code size on 32-bit x86.
      dd5353a4
  26. Dec 17, 2015
  27. Nov 29, 2015
Loading