Skip to content
Snippets Groups Projects
Commit 6f56412e authored by Tom Collins's avatar Tom Collins Committed by Damien George
Browse files

py/lexer: Process CR earlier to allow newlines checks on chr1.

Resolves an issue where lexer failed to accept CR after line continuation
character.  It also simplifies the code.
parent 5feeba88
No related branches found
No related tags found
No related merge requests found
...@@ -137,23 +137,18 @@ STATIC void next_char(mp_lexer_t *lex) { ...@@ -137,23 +137,18 @@ STATIC void next_char(mp_lexer_t *lex) {
lex->chr1 = lex->chr2; lex->chr1 = lex->chr2;
lex->chr2 = lex->reader.readbyte(lex->reader.data); lex->chr2 = lex->reader.readbyte(lex->reader.data);
if (lex->chr0 == '\r') { if (lex->chr1 == '\r') {
// CR is a new line, converted to LF // CR is a new line, converted to LF
lex->chr0 = '\n'; lex->chr1 = '\n';
if (lex->chr1 == '\n') { if (lex->chr2 == '\n') {
// CR LF is a single new line // CR LF is a single new line, throw out the extra LF
lex->chr1 = lex->chr2;
lex->chr2 = lex->reader.readbyte(lex->reader.data); lex->chr2 = lex->reader.readbyte(lex->reader.data);
} }
} }
if (lex->chr2 == MP_LEXER_EOF) { // check if we need to insert a newline at end of file
// EOF, check if we need to insert a newline at end of file if (lex->chr2 == MP_LEXER_EOF && lex->chr1 != MP_LEXER_EOF && lex->chr1 != '\n') {
if (lex->chr1 != MP_LEXER_EOF && lex->chr1 != '\n') { lex->chr2 = '\n';
// if lex->chr1 == '\r' then this makes a CR LF which will be converted to LF above
// otherwise it just inserts a LF
lex->chr2 = '\n';
}
} }
} }
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment