| Age | Commit message (Collapse) | Author |
|---|
| |
| `rb_addrinfo_t` has `VALUE inspectname` and `VALUE canonname`, so this triggers the write barrier for those. The `AddrInfo` wasn't readily available where we need to call `RB_OBJ_WRITE`, so this involves passing `self` around a bit. Notes: Merged: https://.com/ruby/ruby/pull/13503 |
| Allow Addrinfo objects to be shared among Ractors. Addrinfo objects are already immutable, so I think it's safe for us to tag them as RUBY_TYPED_FROZEN_SHAREABLE shareable too. Notes: Merged: https://.com/ruby/ruby/pull/13388 |
| This change addresses the following ASAN error: ``` ==36597==ERROR: AddressSanitizer: heap-use-after-free on address 0x512000396ba8 at pc 0x7fcad5cbad9f bp 0x7fff19739af0 sp 0x7fff19739ae8 WRITE of size 8 at 0x512000396ba8 thread T0 [643/756] 36600=optparse/test_summary #0 0x7fcad5cbad9e in free_fast_fallback_getaddrinfo_entry /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/raddrinfo.c:3046:22 #1 0x7fcad5c9fb48 in fast_fallback_inetsock_cleanup /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/ipsocket.c:1179:17 #2 0x7fcadf3b611a in rb_ensure /home/runner/work/ruby-dev-builder/ruby-dev-builder/eval.c:1081:5 #3 0x7fcad5c9b44b in rsock_init_inetsock /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/ipsocket.c:1289:20 #4 0x7fcad5ca22b8 in tcp_init /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/tcpsocket.c:76:12 #5 0x7fcadf83ba70 in vm_call0_cfunc_with_frame /home/runner/work/ruby-dev-builder/ruby-dev-builder/./vm_eval.c:164:15 ... ``` A `struct fast_fallback_getaddrinfo_shared` is shared between the main thread and two child threads. This struct contains an array of `fast_fallback_getaddrinfo_entry`. `fast_fallback_getaddrinfo_entry` and `fast_fallback_getaddrinfo_shared` were freed separately, and if `fast_fallback_getaddrinfo_shared` was freed first and then an attempt was made to free a `fast_fallback_getaddrinfo_entry`, a `heap-use-after-free` could occur. This change avoids that possibility by separating the deallocation of the addrinfo memory held by `fast_fallback_getaddrinfo_entry` from the access and lifecycle of the `fast_fallback_getaddrinfo_entry` itself. Notes: Merged-By: shioimm <[email protected]> |
| to avoid conflicts with other functions. This was pointed out in https://.com/ruby/ruby/pull/11653#discussion_r1837356617 , but it was not fixed at that time. Notes: Merged-By: shioimm <[email protected]> |
| Wrap `do_fast_fallback_getaddrinfo` with `rb_thread_prevent_fork` Referencing PR #10864, wrap `do_fast_fallback_getaddrinfo` with `rb_thread_prevent_fork` to avoid fork safety issues. `do_fast_fallback_getaddrinfo` internally uses getaddrinfo(3), leading to fork safety issues, as described in PR #10864. This change ensures that `do_fast_fallback_getaddrinfo` is guarded by `rb_thread_prevent_fork`, preventing fork during its execution and avoiding related issues. Notes: Merged-By: shioimm <[email protected]> |
| Any memory allocated with xmalloc needs to be matched with xfree rather than plain free. Ruby unfortunately redefines strdup to be ruby_strdup, which uses xmalloc so needs to be xfreed. Previously these were mismatched. This commit changes the copy to be an explicit ruby_strdup (to avoid confusion) and the free to be xfree. Notes: Merged: https://.com/ruby/ruby/pull/12311 |
| * Use `rb_thread_fd_select` instead of select(2) For fixing https://bugs.ruby-lang.org/issues/20932 . `TCPSocket.new`, which internally uses select(2) for HEv2, can cause SEGV if the number of file descriptors exceeds `FD_SETSIZE`. This change avoids that issue by replacing select(2) with `rb_thread_fd_select`, which is provided as part of Ruby's internal API. --- This includes the following changes. * rb_thread_fd_select does not need common pipe Notes: Merged-By: shioimm <[email protected]> |
| In some locations we were using shared->lock and in others &shared->lock, and we were ing the allocated memory. Notes: Merged: https://.com/ruby/ruby/pull/12239 |
| `TCPSocket.new` with HEv2 uses three threads. The last of these threads to exit closed pipes. However, if pipes were open at the end of the main thread, they would . This change avoids this by closing pipes at the end of the main thread. Notes: Merged-By: shioimm <[email protected]> |
| ``` for (int i = 0; i < arg->family_size; i++) { arg->getaddrinfo_entries[i] = allocate_fast_fallback_getaddrinfo_entry(); if (!(arg->getaddrinfo_entries[i])) rb_syserr_fail(errno, "calloc(3)"); ``` If the allocation fails in the second interation, the memory allocated in the first iteration would be . This change prevents the memory by allocating the memory in advance. (The struct name `fast_fallback_getaddrinfo_shared` might no longer be good.) Notes: Merged: https://.com/ruby/ruby/pull/12163 |
| http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5409001 ``` ================================================================= ==3263562==ERROR: AddressSanitizer: stack-use-after-return on address 0x735a8f190da8 at pc 0x735a6f58dabc bp 0x735a639ffd10 sp 0x735a639ffd08 READ of size 4 at 0x735a8f190da8 thread T211 ================================================================= ``` Notes: Merged-By: shioimm <[email protected]> |
| http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5408428 ``` ==3159643==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x796cf8f09041 at pc 0x6539bbf68ded bp 0x796cfadffcf0 sp 0x796cfadff4b8 READ of size 2 at 0x796cf8f09041 thread T13 #0 0x6539bbf68dec in strlen (/tmp/ruby/build/trunk_asan/ruby+0x18edec) (BuildId: cca267c7ae091060e1b82a6b4ed1aeaf00edebab) ``` Notes: Merged: https://.com/ruby/ruby/pull/12089 |
| TCPSocket.new (#11653) * Introduction of Happy Eyeballs Version 2 (RFC8305) in TCPSocket.new This is an implementation of Happy Eyeballs version 2 (RFC 8305) in `TCPSocket.new`. See https://.com/ruby/ruby/pull/11653 1. Background Prior to this implementation, I implemented Happy Eyeballs Version 2 (HEv2) for `Socket.tcp` in https://.com/ruby/ruby/pull/9374. HEv2 is an algorithm defined in [RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305), aimed at improving network connectivity. For more details on the specific cases that HEv2 helps, please refer to https://bugs.ruby-lang.org/issues/20108. 2. Proposal & Outcome This proposal implements the same HEv2 algorithm in `TCPSocket.new`. Since `TCPSocket.new` is used more widely than `Socket.tcp`, this change is expected to broaden the impact of HEv2's benefits. Like `Socket.tcp`, I have also added `fast_fallback` keyword argument to `TCPSocket.new`. This option is set to true by default, enabling the HEv2 functionality. However, users can explicitly set it to false to disable HEv2 and use the previous behavior of `TCPSocket.new`. It should be noted that HEv2 is enabled only in environments where pthreads are available. This specification follows the approach taken in https://bugs.ruby-lang.org/issues/19965 , where name resolution can be interrupted. (In environments where pthreads are not available, the `fast_fallback` option is ignored.) 3. Performance Below is the benchmark of 100 requests to `www.ruby-lang.org` with the fast_fallback option set to true and false, respectively. While there is a slight performance degradation when HEv2 is enabled, the degradation is smaller compared to that seen in `Socket.tcp`. ``` ~/s/build ❯❯❯ ../install/bin/ruby ../ruby/test.rb Rehearsal -------------------------------------------------------- fast_fallback: true 0.017588 0.097045 0.114633 ( 1.460664) fast_fallback: false 0.014033 0.078984 0.093017 ( 1.413951) ----------------------------------------------- total: 0.207650sec user system total real fast_fallback: true 0.020891 0.124054 0.144945 ( 1.473816) fast_fallback: false 0.018392 0.110852 0.129244 ( 1.466014) ``` * Update debug prints Co-authored-by: Nobuyoshi Nakada <[email protected]> * Remove debug prints * misc * Disable HEv2 in Win * Raise resolution error with hostname resolution * Fix to handle errors * Remove warnings * Errors that do not need to be handled * misc * Improve doc * Fix bug on cancellation * Avoid EAI_ADDRFAMILY for resolving IPv6 * Follow upstream * misc * Refactor connection_attempt_fds management - Introduced allocate_connection_attempt_fds and reallocate_connection_attempt_fds for improved memory allocation of connection_attempt_fds - Added remove_connection_attempt_fd to resize connection_attempt_fds dynamically. - Simplified the in_progress_fds function to only check the size of connection_attempt_fds. * Rename do_pthread_create to raddrinfo_pthread_create to avoid conflicting --------- Co-authored-by: Nobuyoshi Nakada <[email protected]> Notes: Merged-By: shioimm <[email protected]> |
| [Feature #20590] For better of for worse, fork(2) remain the primary provider of parallelism in Ruby programs. Even though it's frowned uppon in many circles, and a lot of literature will simply state that only async-signal safe APIs are safe to use after `fork()`, in practice most APIs work well as long as you are careful about not forking while another thread is holding a pthread mutex. One of the APIs that is known cause fork safety issues is `getaddrinfo`. If you fork while another thread is inside `getaddrinfo`, a mutex may be left locked in the child, with no way to unlock it. I think we could reduce the impact of these problem by preventing in for the most notorious and common cases, by locking around `fork(2)` and known unsafe APIs with a read-write lock. Notes: Merged: https://.com/ruby/ruby/pull/10864 |
| |
| On alpine freeaddrinfo does not accept NULL pointer |
| When the registerred unblock function is called, it should retry the cancelled blocking function if possible after checkints. For example, `SIGCHLD` can cancel this method, but it should not raise any exception if there is no trap handlers. The following is repro-code: ```ruby require 'socket' PN = 10_000 1000000.times{ p _1 PN.times{ fork{ sleep rand(0.3) } } i = 0 while i<PN cpid = Process.wait -1, Process::WNOHANG if cpid # p [i, cpid] i += 1 end begin TCPServer.new(nil, 0).close rescue p $! exit! end end } ``` |
| Previously, EAI_AGAIN was raised. In our CI, "Temporary failure in name resolution" (EAI_AGAIN) is often raised. We are not sure if this was caused by pthread_create failure or getaddrinfo failure. To make it possible to distinguish between them, this changeset raises EAI_SYSTEM instead of EAI_AGAIN on pthread_create failure. |
| |
| |
| In case of EAI_SYSTEM, getaddrinfo is supposed to set more detail in errno; however, because we call getaddrinfo on a thread now, and errno is threadlocal, that information is being lost. Instead, we just raise whatever errno happens to be on the calling thread (which can be something very confusing, like `ECHILD`). Fix it by explicitly propagating errno back to the calling thread through the getaddrinfo_arg structure. [Bug #20198] |
| It looks like `sched_getcpu(3)` returns a strange number on some (virtual?) environments. I decided to remove the setaffinity mechanism because the performance does not appear to degrade on a quick benchmark even if removed. [Bug #20172] |
| [bug #20149] |
| The document must be placed immediately before the class definition. No other statements can be placed in between. |
| Again, rsock_raise_socket_error is called only when getaddrinfo and getaddrname fail |
| According to https://bugs.openjdk.org/browse/JDK-8268605, pthread_create may fail spuriously. This change implements a simple retry as a modest measure, which is also used by JDK. |
| Do not use `pthread_attr_setaffinity_np` if `sched_getcpu()` exceeds `CPU_SETSIZE`. (Using `CPU_ALLOC()` would be more appropriate.) |
| pointed by @nobu |
| |
| After a pthread for getaddrinfo is detached, we cannot predict when the thread will exit. It would lead to a segfault by setting pthread_setaffinity to the terminated pthread. I guess this problem would be more likely to occur in high-load environments. This change detaches the pthread after pthread_setaffinity is called. [Feature #19965] |
| This reverts commit de82439215dd2770ef9a3a2cf5798bdadb788533. |
| Looks like it randomly causes a segfault https://rubyci.s3.amazonaws.com/rhel_zlinux/ruby-master/log/20231025T093302Z.fail.html.gz ``` [11186/26148] TestNetHTTP_v1_2#test_set_form/home/chkbuild/build/20231025T093302Z/ruby/tool/lib/webrick/httprequest.rb:197: [BUG] Segmentation fault at 0x000003ff1ffff000 ruby 3.3.0dev (2023-10-25T07:50:00Z master 526292d9fe) [s390x-linux] ``` |
| |
| Cosmetic change per ko1's preference |
| |
| Same as previous commit for rb_getnameinfo. |
| When pthread_create is available, rb_getaddrinfo creates a pthread and executes getaddrinfo(3) in it. The caller thread waits for the pthread to complete, but detaches it if interrupted. This allows name resolution to be interuppted by Timeout.timeout, etc. even if it takes a long time (for example, when the DNS server does not respond). [Feature #19965] |
| |
| This is a preparation for introducing cancellable getaddrinfo/getnameinfo. |
| |
| |
| * Windows: Fix warning about undefined if_indextoname() * Windows: Fix UNIXSocket on MINGW and make .pair more reliable * Windows: Use nonblock=true for read tests with scheduler * Windows: Move socket detection from File.socket? to File.stat Add S_IFSOCK to Windows and interpret reparse points accordingly. Enable tests that work now. * Windows: Use wide-char functions to UNIXSocket This fixes behaviour with non-ASCII characters. It also fixes deletion of temporary UNIXSocket.pair files. * Windows: Add UNIXSocket tests for specifics of Windows impl. * Windows: fix VC build due to missing _snwprintf Avoid usage of _snwprintf, since it fails linking ruby.dll like so: linking shared-library x64-vcruntime140-ruby320.dll x64-vcruntime140-ruby320.def : error LNK2001: unresolved external symbol snwprintf x64-vcruntime140-ruby320.def : error LNK2001: unresolved external symbol vsnwprintf_l whereas linking miniruby.exe succeeds. This uses snprintf on the UTF-8 string instead. Also remove branch GetWindowsDirectoryW, since it doesn't work. * Windows: Fix dangling symlink test failures Co-authored-by: Lars Kanis <[email protected]> Notes: Merged-By: ioquatix <[email protected]> |
| |
| IPv6 link local address is fe80::/10 not ff80::/10: https://www.rfc-editor.org/rfc/rfc4291.html Link-Local unicast 1111111010 FE80::/10 2.5.6 IPv6 (deprecated) site local address is fec0::/10 not ffc0::/10: https://www.rfc-editor.org/rfc/rfc3513.html Site-local unicast 1111111011 FEC0::/10 2.5.6 Notes: Merged-By: kou <[email protected]> |
| [Misc #18891] Notes: Merged: https://.com/ruby/ruby/pull/6094 |
| - `www.ruby-lang.org` links to `./www.ruby-lang.org` - `cgi['field_name']` links to `./'field_name'` |
| Co-authored-by: Bruno Sutic <[email protected]> Notes: Merged-By: ioquatix <[email protected]> |
| * Use the wrapper of rb_cObject instead of data access * Replaced rest of extentions * Updated the version guard for Data * Added the version guard of rb_cData Notes: Merged: https://.com/ruby/ruby/pull/3961 |
| getaddrinfo_a() gets stuck after fork(). To avoid this, we need 1 second sleep to wait for internal worker threads of getaddrinfo_a() to be finished, but that is unacceptable. [Bug #17220] [Feature #17134] [Feature #17187] |
