What's New in RubyGems/Bundler 4.1, Part 3: Unifying RubyGems and Bundler, and What gem install No Longer Does

I'm Hiroshi Shibata (hsbt), a Ruby committer and the lead maintainer of RubyGems and Bundler.

TL;DR This is the last part of a three-part series on what's new in RubyGems and Bundler 4.1. Part 1 covered override and prune, and Part 2 covered defenses against supply chain attacks. This part is about bringing RubyGems and Bundler together.

Most of that work is invisible to users, so I'll start from the visible side, which is what gem install stopped doing in 4.1. It no longer fetches a gemspec per installed gem, it caches the compact index and asks with ETags, and its resolver is now PubGrub, the same as Bundler's. Behind that, the two codebases now share one layout, one HTTP stack, and one copy of most vendored libraries. Commands that read the whole index, such as gem search -r, got slower, and that's still open.

The numbers below were measured on October 9, 2026, on macOS (arm64) with Ruby 4.0.7 and YJIT off. I checked out the v4.0.22 and v4.1.0.beta2 tags of ruby/rubygems and instrumented Gem::RemoteFetcher#request to count HTTP requests and response body bytes, with a separate cache directory for each run.

One round trip per installed gem

gem install --no-document activesupport installs ten gems, dependencies included. Here are the HTTP requests it makes.

The ten /quick requests that disappeared match the number of installed gems. Up to 4.0, RubyGems finished resolving and then went back to the server for a Marshal gemspec for each gem. All ten add up to only 6,213 bytes, so they barely show in the byte count, but each one is a round trip you wait for.

They went away because ruby/rubygems#9606 changed how Gem::Resolver::APISpecification#spec works. The compact index data used for resolution already has everything needed to download and install a gem. RubyGems now builds the Gem::Specification from it, so there is no reason left to fetch the Marshal gemspec.

What the compact index holds

The compact index is the index format Bundler has used for years. index.rubygems.org/versions is a text file listing every version of every gem, one per line. index.rubygems.org/info/{gem} is a text file per gem listing its dependencies and the Ruby versions it needs.

gem install in 4.0 already read /info/{gem} when talking to rubygems.org. What changed in 4.1 is a disk cache on top of it. Gem::CompactIndexClient stores each file's ETag and sends If-None-Match the next time.

The difference is clearest with gem install --explain rails, which only resolves, run repeatedly against the same cache.

4.0 downloaded all 83 /info/{gem} files again with a 200 every time. The 66 that 4.1 asks for all come back as 304 Not Modified, with no body at all. Alternating the two 15 times each, the median went from 1.088 seconds to 0.904 seconds. A 304 is still a round trip, so the time saved is less dramatic than the bytes saved.

The cost of replacing the full index

gem update, gem outdated, gem list -r, and gem search -r need the whole index. Up to 4.0 they fetched rubygems.org's Marshal index, about 1.7MB for latest_specs.4.8.gz and about 6.3MB for specs.4.8.gz with every version.

#9606 moved this path to the compact index too. The versions file, though, is about 23MB, much larger than the Marshal index. I ran gem search -r rails three times each with an empty cache.

4.0.22 fetched 1,676,090 bytes with a median of 1.461 seconds. 4.1.0.beta2 fetched 23,394,089 bytes with a median of 4.573 seconds.

Transfer pays off from the second run. The Marshal index has no way to fetch a diff. When it has been updated, If-Modified-Since gets a 200 and the whole file comes down again. With the cache set back one day, 4.0.22 downloaded 1,675,927 bytes again.

The compact index asks only for the tail with a Range header. With the cache truncated back to October 1, the 206 Partial Content response was 359,487 bytes.

rubygems.org rebuilds versions once a month. The current file was built on October 1, 2026, and in the eight-odd days since, about 360KB, or 6,194 lines, had been appended. That's about 43KB a day. If you update the index daily, 4.1 transfers 43KB where 4.0 transfers 1.68MB.

The first update after a rebuild is the exception. The local cache no longer matches, so the whole 23MB comes down again. Even counting that, a month of daily updates transfers about half of what 4.0 does.

Run time is another story. Even with a 304 and zero bytes transferred, gem search -r rails got slower. Over six runs each that got a 304, the median was 1.392 seconds for 4.0.22 and 3.877 seconds for 4.1.0.beta2.

4.1 reads every line of versions and builds a Gem::Version for every version before narrowing down, even when only the latest is wanted. For commands that read the whole index, less transfer came at the price of longer runs. There's room to improve here.

The resolver is now PubGrub

In the measurements above, the number of /info/{gem} fetches dropped from 19 to 14 for activesupport and from 83 to 66 for rails. That comes from ruby/rubygems#9402, which replaced the gem command's resolver, Molinillo, with PubGrub. Bundler has used PubGrub for a while, and RubyGems was the only one still carrying Molinillo.

The search order differs, so the number of candidates asked about before reaching a solution differs too. The 17 that only 4.0 asked about while resolving rails include old gems such as actionwebservice and rack-ssl that never end up in the final dependencies. PubGrub reaches a solution without asking about them.

Moving duplicated code to one side

Now for the unification itself. RubyGems and Bundler are developed in the same repository, but up to 4.0 everything under bundler/ was its own tree, with a separate lib, exe, and gemspec for Bundler. ruby/rubygems#9634 moved it to the top of the repository, touching 387 files. ruby/ruby has long imported Bundler with a flat layout, so syncing no longer needs manual fixups each time.

With one layout, code that both sides kept separately can move to one side. First, vendored libraries.

Only fileutils and thor are left. uri (#9650), securerandom (#9651), and pub_grub (#9752) now use the RubyGems copy. connection_pool and net-http-persistent went away together in #9786. Bundler used net-http-persistent only to fetch metadata and Gem::RemoteFetcher to download the gems themselves, which meant two HTTP stacks. Unifying them on Gem::Request removed about 2,200 lines.

It's not only vendored code. Implementations we wrote ourselves moved too, the YAML serializer in #9751 and the compact index client in #9753. Gem::CompactIndexClient and Bundler::CompactIndexClient were nearly identical, and every fix had to go into both.

It also goes the other way. #9564 dropped the RubyGems lockfile parser in favor of Bundler::LockfileParser. Which side wins is decided per implementation, and the direction isn't fixed.

The PRs that moved code to the RubyGems copy also ship that copy inside the Bundler gem. Bundler has to run on older RubyGems, so it doesn't break on a RubyGems that has no Gem::CompactIndexClient.

The documents became one set too. #9851 left one README and one MIT license, and #9847 merged the changelogs. The latter moved 21 Bundler sections since 4.0.0, 197 entries, under a ### Bundler heading in the matching release.

Smaller fixes along the way

Apart from the unification, 4.1 has a few smaller fixes.

#9093 removed Gem::Package#verify_gz. Before extracting a gem, it read all of data.tar.gz to check the gzip checksum. The extract_files that follows already fails when it can't inflate what it reads, so verification alone meant reading the file twice. In the reporter's profile, it took 9% to 17% of the install thread for gems without native extensions or plugins.

#9689 is about Bundler's git sources. When several gems come from one git block, cache_to, which writes to vendor/cache, ran once per gem and checked out the whole repository each time. Remembering whether a path was already written during the same command is enough, and it helps most when you point at a large repository such as rails.

#9339 fixes concurrent_downloads in .gemrc being ignored. Gem::ConfigFile in 4.0 still had a TODO saying it should be handled like the other options, and it wasn't. The default of 8 was always used, so environments that wanted fewer parallel downloads couldn't get them.

Try It

The unification in 4.1 is mostly about moving code written twice to one side. What users see is fewer round trips in gem install and index updates that fetch only the diff. On the other hand, commands that read the whole index, like gem search -r, got slower. We'll keep treating RubyGems and Bundler as one product. How far to take the repository structure and the shared code isn't settled yet, and I'll write again when it is.

4.1.0.beta3 is on rubygems.org, so you can try it today.

gem update --system 4.1.0.beta3
gem install bundler -v 4.1.0.beta3

If anything behaves oddly, please report it to ruby/rubygems.

Notes

The measurements were taken on October 9, 2026, against v4.0.22 and v4.1.0.beta2. Between beta2 and beta3 the index code only changed to skip blank lines in versions, and the vendored libraries are the same. The rest was checked against 4.1.0.beta3 in November 2026. Details may change before the final release.