Change bundle install with a lockfile to respect the BUNDLED WITH bundler version

[duckinator]

This implements part of the Bundler Version Locking RFC.

What was the end-user or developer problem that led to this PR?

It's hard to ensure that all developers and environments of an application use the exact same bundler version. There's many reasons why that's a good thing, for example, wanting to enforce that a security release is used, so that nobody is vulnerable to a security issue.

What is your fix for the problem, implemented in this PR?

My fix is to respect the BUNDLED WITH version if there is a lockfile. This section was originally introduced for this exact purpose but was never actually enforced.

The strategy is that when bundle install is run, if the running version does not match the BUNDLED WITH version, bundler will first install the BUNDLED WITH version, and then re-exec using that version.

This strategy is implemented as conservatively as we could:

• Other versions can still be specific with the BUNDLER_VERSION environment variable or with bundle _<version>_.
• If anything goes wrong when installing the BUNDLED WITH version, bundler will go on using the running version as a fallback.
• This feature is only applied when bundler is run in combination with the latest rubygems.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[duckinator]

Either way, here's a screenshot showing this PR working! :3

[duckinator]

Fixed some linting failures. (I'm having trouble getting rubocop to run locally, so had to rely on CI for that, heh.)

[duckinator]

I fixed the linter errors but introduced new ones, the first time. I think it's right, now. 😂

[duckinator]

I have no idea what's causing those CI errors; will have to look into it another day.

[duckinator]

Getting back into this again. Rebased off master.

[duckinator]

Rebased off master. Also realized I finished an item from the TODO checklist at some point already, so checked that off.

[duckinator]

I may be able to switch from Kernel.exec(...) to Bundler.original_exec(...). According to the function documentation, this should address the problem @simi mentioned in rubygems/rfcs#29 (comment):

https://github.com/rubygems/rubygems/blob/9c88db949d7480ebea4cae29dfad1a5ad6db8cd1/bundler/lib/bundler.rb#L422-L425

[duckinator]

I'm a bit stuck atm because I'm not sure how bundle _VERSION_ ... is implemented, and can't find it from poking around.

[simi]

@duckinator that's feature of generated binstub by gem installer.

https://github.com/rubygems/rubygems/blob/5b910cc4a78827c0571252dd93776b559cef608a/lib/rubygems/installer.rb#L214-L231

example of rails binstub

[retro@retro  git]❤ cat ~/.gem/ruby/3.0.2/bin/rails 
#!/usr/bin/env ruby
#
# This file was generated by RubyGems.
#
# The application 'railties' is installed as part of a gem, and
# this file is here to facilitate running it.
#

require 'rubygems'

version = ">= 0.a"

str = ARGV.first
if str
  str = str.b[/\A_(.*)_\z/, 1]
  if str and Gem::Version.correct?(str)
    version = str
    ARGV.shift
  end
end

if Gem.respond_to?(:activate_bin_path)
load Gem.activate_bin_path('railties', 'rails', version)
else
gem "railties", version
load Gem.bin_path("railties", "rails", version)
end

[duckinator]

Huh, okay, so there's no actual way to determine whether it was started using that mechanism or not. That makes me wonder if this all needs to be dragged up to the Bundler binstub.

Thanks for the info, @simi. I'll have to think a bit on how to approach this.

[deivid-rodriguez]

@duckinator I'm taking over this PR now if you don't mind.

[duckinator]

@deivid-rodriguez sorry for the delay in replying, but go for it. 👍

[deivid-rodriguez]

Alright, just to document progress here. Things are green and I'm pretty satisfied with this as an initial version. The current approach is to parse the BUNDLED WITH version, install it and Kernel.exec to it unless:

• There's no lockfile or no BUNDLED WITH section inside the lockfile.
• The running version matches the BUNDLED WITH version exactly.
• The lockfile is locked to a development version of bundler.
• An explicit BUNDLER_VERSION is set on the environment.

Another exception is if bundle _<version>_ has been passed explicitly in the command line. However, binstubs completely remove that argument when they process it, so we have no way to know whether _<version>_ has been passed originally. The solution I came up with for that is to change the bundler binstub to explicitly set BUNDLER_VERSION when _<version>_ is given. That way this case falls under the fourth bullet point above. It also prompted me to require the latest version of rubygems (the one that generates this modified bundler binstubs) for this feature to be enabled.

In the next couple of days I want to wrap this up with:

• Some more tests.
• Rescue any error that I might've missed when installing bundler, and go on with the running bundler in that case (with some logging of the error I guess).
• Detect the situation where the running bundler location is not writable, and skip the feature there too, because we will try to install the new version alongside with the running bundler version and that will fail. I hope this will seamlessly skip this feature when using OS packaged rubies.
• Investigate why I needed b398f80. (Fixed at 5ed9d47).

[deivid-rodriguez]

Funny how normally tests are supposed to catch incorrect implementations. In this case, a correct implementation caught two bugs in the tests 😄.

[ngan]

this is awesome. Thanks @duckinator and @deivid-rodriguez

[duckinator]

@deivid-rodriguez thank you for making sure this PR got wrapped up and merged. I appreciate it a lot!

[ysbaddaden]

Is it possible to DISABLE or OPT-OUT from that feature?

Bundler keeps on downgrading itself in my Docker images. I just want to use the bundler that is already available, not downgrade to the version from an older image 😭

[simi]

Is it possible to DISABLE or OPT-OUT from that feature?

Bundler keeps on downgrading itself in my Docker images. I just want to use the bundler that is already available, not downgrade to the version from an older image sob

The proper way to handle is to update the bundler in Gemfile.lock to the one you use, in your case to the one you have in docker image.