Berkshelf Integration

[hansnqyr]

Berkshelf and Knife both maintain different (and often conflicting) gem dependencies.
This isn't entirely surprising as Berkshelf is something of a knife replacement.

Given the above, I'm beginning to think that berkshelf integration in Knife-Spork is a unicorn.

[jonlives]

@hanskreuger do you have any further information on the conflicting dependencies? I'm pretty hopeful any issues can be ironed out, there isn't too much in spork that's dependent on specific gem versions.

[hansnqyr]

@jonlives in this case it was json - the issue here isn't within spork proper, but rather the fact that we're trying to pull berkshelf into a knife plugin.
I'm happy to go into specific details of this instance, but I opened this issue to discuss the general approach.

[lamont-granquist]

On any knife command (not just spork) the subcommand loader will load spork which will load up Berkshelf, which will drag all its deps into the process, which is totally awful:

% knife gem install
ont/.rvm/gems/ruby-2.1.3/gems/buff-config-1.0.1/lib/buff/config/ruby.rb:30:in `rescue in initialize': uninitialized constant #<Class:0x007fe9d1df9f68>::Chef::REST (Buff::Errors::InvalidConfig)
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/buff-config-1.0.1/lib/buff/config/ruby.rb:25:in `initialize'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/buff-config-1.0.1/lib/buff/config/ruby.rb:17:in `new'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/buff-config-1.0.1/lib/buff/config/ruby.rb:17:in `parse'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/buff-config-1.0.1/lib/buff/config/ruby.rb:119:in `from_ruby'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/buff-config-1.0.1/lib/buff/config/ruby.rb:112:in `initialize'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/ridley-4.1.0/lib/ridley/chef/config.rb:83:in `initialize'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf.rb:90:in `new'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf.rb:90:in `chef_config'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf/config.rb:80:in `<class:Config>'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf/config.rb:4:in `<module:Berkshelf>'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf/config.rb:3:in `<top (required)>'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf/cli.rb:2:in `require_relative'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf/cli.rb:2:in `<top (required)>'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf.rb:195:in `require_relative'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/berkshelf-3.2.1/lib/berkshelf.rb:195:in `<top (required)>'
    from /Users/lamont/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `require'
    from /Users/lamont/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
    from /Users/lamont/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:144:in `require'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/knife-spork-1.4.2/lib/chef/knife/spork-omni.rb:3:in `<top (required)>'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/lib/chef/knife/core/subcommand_loader.rb:40:in `load'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/lib/chef/knife/core/subcommand_loader.rb:40:in `block in load_commands'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/lib/chef/knife/core/subcommand_loader.rb:40:in `each'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/lib/chef/knife/core/subcommand_loader.rb:40:in `load_commands'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/lib/chef/knife.rb:122:in `load_commands'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/lib/chef/knife.rb:203:in `run'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/lib/chef/application/knife.rb:139:in `run'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/gems/chef-12.2.0.dev.0/bin/knife:25:in `<top (required)>'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/bin/knife:23:in `load'
    from /Users/lamont/.rvm/gems/ruby-2.1.3/bin/knife:23:in `<main>'

There is a bug there somewhere, but the entire stacktrace popping up when I'm not even doing a knife-spork command is a much larger bug.

One thing you could do would be to lazily load berkshelf in the deps method of the knife subcommand rather than doing that up front. The other thing to do would just be to not 'require' berkshelf at all and simply shell out to doing berks command. I don't quite know what you're doing with berkshelf right now, but if its just a 'berks upload' then if you just fork and run berks upload you avoid all the gem dep hell pain by spawning a new Ruby VM. Which is both horrific and yet neatly punts all kinds of gem dep hell into to the 'no fucks given' bucket. If berks wants to load up a completely different version of chef to get its work done from the one that is running knife-spork it could do something that insane. If you can't get what you need off the berks cli then I'd actually suggest adding some cli options to berks so that you could. That would be optimally lazy and would completely decouple the gem deps.