| Age | Commit message (Collapse) | Author |
|---|
| |
| No parser semantic value types are `VALUE` then no need to use imemo for managing semantic value stack anymore. |
| clause duplication This commit simplifies warnings for hash keys duplication and when clause duplication, based on the discussion of https://bugs.ruby-lang.org/issues/20331. Warnings are reported only when strings are same to ohters. |
| |
| The size of `struct parser_params` is 8 bytes difference in `ripper_s_allocate` and `rb_ruby_parser_allocate` when the universal parser is enabled. This causes a situation where `*r->p` is not fully initialized in `ripper_s_allocate` as shown below. ```console (gdb) p *r->p $2 = {heap = 0x0, lval = 0x0, yylloc = 0x0, lex = {strterm = 0x0, gets = 0x0, input = 0, string_buffer = {head = 0x0, last = 0x0}, lastlin e = 0x0, nextline = 0x0, pbeg = 0x0, pcur = 0x0, pend = 0x0, ptok = 0x0, gets_ = {ptr = 0, call = 0x0}, state = EXPR_NONE, paren_nest = 0, lpar _seen = 0, debug = 0, has_shebang = 0, token_seen = 0, token_info_enabled = 0, error_p = 0, cr_seen = 0, value = 0, result = 0, parsing_thread = 0, s_value = 0, s_lvalue = 0, s_value_stack = 2097} ```` This seems to cause `double free or corruption (!prev)` and SEGV. So, fixing this by introduce `rb_ripper_parser_params_allocate` and `rb_ruby_parser_config` functions for Ripper, and `struct parser_params` same size is returned. |
| - Introduce `rb_parser_ary_t` structure to partly eliminate RArray from parse.y - In this , `parser_params->tokens` and `parser_params->ast->node_buffer->tokens` are now `rb_parser_ary_t *` - Instead, `ast_node_all_tokens()` internally creates a Ruby Array object from the `rb_parser_ary_t` - Also, delete `rb_ast_tokens()` and `rb_ast_set_tokens()` in node.c - Implement `rb_parser_str_escape()` - This is a port of the `rb_str_escape()` function in string.c - `rb_parser_str_escape()` does not depend on `VALUE` (RString) - Instead, it uses `rb_parser_stirng_t *` - This function works when --dump=y option passed - Because WIP of the universal parser, similar functions like `rb_parser_tokens_free()` exist in both node.c and parse.y. Refactoring them may be needed in some way in the future - Although we considered redesigning the structure: `ast->node_buffer->tokens` into `ast->tokens`, we leave it as it is because `rb_ast_t` is an imemo. (We will address it in the future) |
| This commit changes rb_fstring to rb_str_to_interned_str in parse.y. rb_fstring is private so it shouldn't be used by ripper. |
| Rather than exposing that an imemo has a flag and four fields, this changes the implementation to only expose one field (the klass) and fills the rest with 0. The type will have to fill in the values themselves. |
| |
| |
| |
| |
| Co-authored-by: Yuichiro Kaneko <[email protected]> |
| |
| For example: 10.times do 100_000.times do eval('{"\xC3": 1}') rescue EncodingError end puts `ps -o rss= -p #{$$}` end Before: 32032 48464 66112 84192 100592 117520 134096 150656 167168 183760 After: 17120 17120 17120 17120 18560 18560 18560 18560 18560 18560 |
| `dest` of `const_decl_path` is `NODE_COLON2` or `NODE_COLON3` in some cases. For example, `B::C ||= [“Not ” + “shareable”]` passes `NODE_COLON2` and `::C ||= [“Not ” + “shareable”]` passes `NODE_COLON3`. This commit fixes `Ractor::IsolationError` message for such case. ``` # shareable_constant_value: literal ::C ||= ["Not " + "shareable"] # Before # => cannot assign unshareable object to C (Ractor::IsolationError) # After # => cannot assign unshareable object to ::C (Ractor::IsolationError) ``` |
| If lhs of assignment is top-level constant reference, the first constant name is omitted from error message. This commit fixes it. ``` # shareable_constant_value: literal ::C = ["Not " + "shareable"] # Before # => cannot assign unshareable object to (Ractor::IsolationError) # After # => cannot assign unshareable object to ::C (Ractor::IsolationError) ``` |
| String nodes holds ruby string object on `VALUE nd_lit`. This commit changes it to `struct rb_parser_string *string` to reduce dependency on ruby object. Sometimes these strings are concatenated with other string therefore string concatenate functions are needed. |
| Introduce `rb_node_const_decl_val` function to allow `rb_ary_join` and `rb_ary_reverse` functions to be removed from Universal Parser. |
| `__ENCODING__ `was managed by `NODE_LIT` with Encoding object. Introduce `NODE_ENCODING` for 1. `__ENCODING__` is detectable from AST Node. 2. Reduce dependency Ruby object for parse.y |
| This commit changes `struct parser_params` lastline and nextline from `VALUE` (String object) to `rb_parser_string_t *` so that dependency on Ruby Object is reduced. `parser_string_buffer_t string_buffer` is added to `struct parser_params` to manage `rb_parser_string_t` pointers of each line. All allocated line strings are freed in `rb_ruby_parser_free`. |
| |
| |
| They don't compile nodes then remove compile_ prefix. `compile_numeric_literal` always returns integer then use integer instead of numeric. |
| This existed before 1b8d01136c3ff6c60325c7609d61e19ac42acd9f. |
| |
| It's allocated outside of parser then no need to track reference count in rb_parser_config. |
| |
| |
| `:sym` was managed by `NODE_LIT` with `Symbol` object. This commit introduces `NODE_SYM` so that 1. Symbol literal is detectable from AST Node 2. Reduce dependency on ruby object |
| Change the argument to align with other node value functions like `rb_node_line_lineno_val`. |
| |
| |
| |
| `__FILE__` was managed by `NODE_STR` with `String` object. This commit introduces `NODE_FILE` and `struct rb_parser_string` so that 1. `__FILE__` is detectable from AST Node 2. Reduce dependency ruby object |
| `__LINE__` was managed by `NODE_LIT` with `Integer` object. This commit introduces `NODE_LINE` so that 1. `__LINE__` is detectable from AST Node 2. Reduce dependency ruby object |
| |
| |
| |
| |
| |
| All kind of AST nodes use same struct RNode, which has u1, u2, u3 union members for holding different kind of data. This has two problems. 1. Low flexibility of data structure Some nodes, for example NODE_TRUE, don’t use u1, u2, u3. On the other hand, NODE_OP_ASGN2 needs more than three union members. However they use same structure definition, need to allocate three union members for NODE_TRUE and need to separate NODE_OP_ASGN2 into another node. This change removes the restriction so make it possible to change data structure by each node type. 2. No compile time check for union member access It’s developer’s responsibility for using correct member for each node type when it’s union. This change clarifies which node has which type of fields and enables compile time check. This commit also changes node_buffer_elem_struct buf management to handle different size data with alignment. |
| NODE_ARGS, NODE_ARYPTN, NODE_FNDPTN manage memory of their structure by imemo tmpbuf Object. However rb_ast_struct has reference to NODE. Then these memory can be freed directly when rb_ast_struct is freed. This commit reduces parser's dependency on CRuby functions. |
| Notes: Merged: https://.com/ruby/ruby/pull/8292 |
| Notes: Merged: https://.com/ruby/ruby/pull/8289 |
| Notes: Merged: https://.com/ruby/ruby/pull/8289 |
| Notes: Merged: https://.com/ruby/ruby/pull/8190 |
| Notes: Merged: https://.com/ruby/ruby/pull/8170 |
| Notes: Merged: https://.com/ruby/ruby/pull/8044 |
| Notes: Merged: https://.com/ruby/ruby/pull/8029 |
