| Age | Commit message (Collapse) | Author |
|---|
| Notes: Merged: https://.com/ruby/ruby/pull/13439 |
| Notes: Merged: https://.com/ruby/ruby/pull/13154 |
| While profiling msgpack-ruby I noticed a very substantial amout of time spent in `rb_enc_associate_index`, called by `rb_utf8_str_new`. On that benchmark, `rb_utf8_str_new` is 33% of the total runtime, in big part because it cause GC to trigger often, but even then `5.3%` of the total runtime is spent in `rb_enc_associate_index` called by `rb_utf8_str_new`. After closer inspection, it appears that it's performing a lot of safety check we can assert we don't need, and other extra useless operations, because strings are first created and filled as ASCII-8BIT and then later reassociated to the desired encoding. By directly allocating the string with the right encoding, it allow to skip a lot of duplicated and useless operations. After this change, the time spent in `rb_utf8_str_new` is down to `28.4%` of total runtime, and most of that is GC. Notes: Merged: https://.com/ruby/ruby/pull/12076 |
| |
| [Bug #20598] Just like [Bug #20595], Encoding#name_list and Encoding#aliases can have their strings corrupted when Encoding.default_internal is set to nil. Co-authored-by: Matthew Valentine-House <[email protected]> |
| [Bug #20595] enc_set_default_encoding will free the C string if the encoding is nil, but the C string can be used by the encoding name string. This will cause the encoding name string to be corrupted. Consider the following code: Encoding.default_internal = Encoding::ASCII_8BIT names = Encoding.default_internal.names p names Encoding.default_internal = nil p names It outputs: ["ASCII-8BIT", "BINARY", "internal"] ["ASCII-8BIT", "BINARY", "\x00\x00\x00\x00\x00\x00\x00\x00"] Co-authored-by: Matthew Valentine-House <[email protected]> |
| [Feature #18576] Since outright renaming `ASCII-8BIT` is deemed to backward incompatible, the next best thing would be to only change its `#inspect`, particularly in exception messages. |
| This `st_table` is used to both mark and pin classes defined from the C API. But `vm->mark_object_ary` already does both much more efficiently. Currently a Ruby process starts with 252 rooted classes, which uses `7224B` in an `st_table` or `2016B` in an `RArray`. So a baseline of 5kB saved, but since `mark_object_ary` is preallocated with `1024` slots but only use `405` of them, it's a net `7kB` save. `vm->mark_object_ary` is also being refactored. Prior to this changes, `mark_object_ary` was a regular `RArray`, but since this allows for references to be moved, it was marked a second time from `rb_vm_mark()` to pin these objects. This has the detrimental effect of marking these references on every minors even though it's a mostly append only list. But using a custom TypedData we can save from having to mark all the references on minor GC runs. Addtionally, immediate values are now ignored and not appended to `vm->mark_object_ary` as it's just wasted space. |
| There is a memory in Encoding.default_external= and Encoding.default_internal= because the duplicated name is not freed when overwriting. 10.times do 1_000_000.times do Encoding.default_internal = nil end puts `ps -o rss= -p #{$$}` end Before: 25664 41504 57360 73232 89168 105056 120944 136816 152720 168576 After: 9648 9648 9648 9680 9680 9680 9680 9680 9680 9680 |
| when the RUBY_FREE_ON_SHUTDOWN environment variable is set, manually free memory at shutdown. Co-authored-by: Nobuyoshi Nakada <[email protected]> Co-authored-by: Peter Zhu <[email protected]> |
| It doesn't even have a mark function. It's only about a hundred objects, but not reason to scan them every time. Notes: Merged: https://.com/ruby/ruby/pull/7254 |
| Notes: Merged: https://.com/ruby/ruby/pull/7079 |
| now `enc_table->list` is not a pointer. |
| This reduces global lock acquiring for reading. https://bugs.ruby-lang.org/issues/18949 Notes: Merged: https://.com/ruby/ruby/pull/6935 |
| UTF-16/UTF-32 * And simplify callers of get_actual_encoding(). * See [Feature #18949]. * See https://.com/ruby/ruby/pull/6322#issuecomment-1242758474 |
| * See [Feature #18949]. |
| [Misc #18891] Notes: Merged: https://.com/ruby/ruby/pull/6094 |
| Otherwise it's way too easy to confuse it with US_ASCII. Notes: Merged: https://.com/ruby/ruby/pull/6127 |
| Main doc for encodings moved from encoding.c to doc/encodings.rdoc Notes: Merged-By: BurdetteLamar <[email protected]> |
| Notes: Merged: https://.com/ruby/ruby/pull/5015 |
| Less macros == huge win. Notes: Merged: https://.com/ruby/ruby/pull/4909 |
| Instead of relying on setting an unsetting ruby_verbose, which is not thread-safe, restructure require_internal and load_lock to accept a warn argument for whether to warn, and add rb_require_internal_silent to require without warnings. Use rb_require_internal_silent when loading encoding. Note this does not modify ruby_debug and errinfo handling, those remain thread-unsafe. Also silent requires when loading transcoders. Notes: Merged: https://.com/ruby/ruby/pull/4887 |
| Notes: Merged: https://.com/ruby/ruby/pull/4759 |
| Notes: Merged: https://.com/ruby/ruby/pull/4695 Merged-By: nobu <[email protected]> |
| If called with an autoloaded encoding that was not yet initialized, `rb_enc_interned_str` would crash with a NULL pointer exception. See: https://.com/ruby/ruby/pull/4119#issuecomment-800189841 Notes: Merged: https://.com/ruby/ruby/pull/4290 |
| rb_enc_from_index(index) doesn't need locking if index specify ASCII/US_ASCCII/UTF-8. rb_enc_from_index() is called frequently so it has impact. user system total real r_parallel/miniruby 174 0.000209 0.000000 5.559872 ( 1.811501) r_parallel/master_mini 175 0.000238 0.000000 12.664707 ( 3.523641) (repeat x1000 `s.split(/,/)` where s = '0,,' * 1000) Notes: Merged: https://.com/ruby/ruby/pull/3916 |
| * Use UTF-8 as default for Encoding.default_external on Windows * Document UTF-8 change on Windows to Encoding.default_external fix https://bugs.ruby-lang.org/issues/16604 Notes: Merged-By: nurse <[email protected]> |
| To make some kind of Ractor related extensions, some functions should be exposed. * include/ruby/thread_native.h * rb_native_mutex_* * rb_native_cond_* * include/ruby/ractor.h * RB_OBJ_SHAREABLE_P(obj) * rb_ractor_shareable_p(obj) * rb_ractor_std*() * rb_cRactor and rm ractor_pub.h and rename srcdir/ractor.h to srcdir/ractor_core.h (to avoid conflict with include/ruby/ractor.h) Notes: Merged: https://.com/ruby/ruby/pull/3775 |
| Notes: Merged: https://.com/ruby/ruby/pull/3589 |
| This reverts commit de17e2dea137bc5ba9f00e3acec32792d0dbb2eb. This can introduce race condition because of conflicting read/write access for enc_table::default_list. Maybe we need to freeze default_list at the end of Init_encdb() in enc/encdb.c. |
| To reduce the number of locking for encoding manipulation, enc_table::list is splited to ::default_list and ::additional_list. ::default_list is pre-allocated and no need locking to access to the ::default_list. If additional encoding space is needed, use ::additional_list and this list need to use locking. However, most of case, ::default_list is enough. Notes: Merged: https://.com/ruby/ruby/pull/3664 |
| https://hackerone.com/reports/954433 |
| Encoding objects can be accessed in multi-ractors and there is no state to mutate. So we can mark it as frozen and shareable. [Bug #17188] Notes: Merged: https://.com/ruby/ruby/pull/3654 |
| enc_table which manages Encoding information. rb_encoding_list also manages Encoding objects. Both are accessed/modified by ractors simultaneously so that they should be synchronized. For enc_table, this introduced GLOBAL_ENC_TABLE_ENTER/LEAVE/EVAL to access this table with VM lock. To make shortcut, three new global variables global_enc_ascii, global_enc_utf_8, global_enc_us_ascii are also introduced. For rb_encoding_list, we split it to rb_default_encoding_list (256 entries) and rb_additional_encoding_list. rb_default_encoding_list is fixed sized Array so we don't need to synchronized (and most of apps only needs it). To manage 257 or more encoding objects, they are stored into rb_additional_encoding_list. To access rb_additional_encoding_list., VM lock is needed. Notes: Merged: https://.com/ruby/ruby/pull/3654 |
| Source code encoding is defaulted to UTF-8 now too. |
| To fix build failures. Notes: Merged: https://.com/ruby/ruby/pull/3079 |
| This shall fix compile errors. Notes: Merged: https://.com/ruby/ruby/pull/3079 |
| |
| Split ruby.h Notes: Merged-By: shyouhei <[email protected]> |
| After the encoding index instance variable is removed when all instance variables are removed in `obj_free`, then `rb_str_free` causes uninitialized instance variable warning and nil-to-integer conversion exception. Both cases result in object allocation during GC, and crashes. |
| This `str2` variable can be non-string (regexp etc.) but the previous code passed it directly to rb_enc_str_asciionly_p(), which expects its argument be a string. Let's enforce that constraint. |
| Function pointers are not void*. See also ce4ea956d24eab5089a143bba38126f2b11b55b6 8427fca49bd85205f5a8766292dd893f003c0e48 |
| Data written to files is not transcoded per default, but only when default_internal is set. The default for default_internal is nil and doesn't depend on the source file encoding. Notes: Merged: https://.com/ruby/ruby/pull/2878 |
| Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies). Notes: Merged: https://.com/ruby/ruby/pull/2711 |
| Notes: Merged: https://.com/ruby/ruby/pull/2760 |
| Notes: Merged: https://.com/ruby/ruby/pull/2760 |
| This removes the related tests, and puts the related specs behind version guards. This affects all code in lib, including some libraries that may want to support older versions of Ruby. Notes: Merged: https://.com/ruby/ruby/pull/2476 |
| This removes the security features added by $SAFE = 1, and warns for access or modification of $SAFE from Ruby-level, as well as warning when calling all public C functions related to $SAFE. This modifies some internal functions that took a safe level argument to no longer take the argument. rb_require_safe now warns, rb_require_string has been added as a version that takes a VALUE and does not warn. One public C function that still takes a safe level argument and that this doesn't warn for is rb_eval_cmd. We may want to consider adding an alternative method that does not take a safe level argument, and warn for rb_eval_cmd. Notes: Merged: https://.com/ruby/ruby/pull/2476 |
| |
| This changeset basically replaces `ruby_xmalloc(x * y)` into `ruby_xmalloc2(x, y)`. Some convenient functions are also provided for instance `rb_xmalloc_mul_add(x, y, z)` which allocates x * y + z byes. Notes: Merged: https://.com/ruby/ruby/pull/2540 |
