napa + npm shrinkwrap incompatible?

[kud]

"napa": {
    "malihu-scrollbar": "malihu/malihu-custom-scrollbar-plugin#2.8.3",
    "bxslider": "wandoledzep/bxslider-4#798eda1d14b108d8be8f0ab3ec2a2fb4a41ed6d7",
    "swfobject": "swfobject/swfobject#a22b7db077abc126d6aa5f2d0f44b11e4ed97940",
    "hint": "chinchang/hint.css#v1.3.1",
    "colorbox": "jackmoore/colorbox#1.5.9",
    "jquery-hashchange": "cowboy/jquery-hashchange#0310f3847f9010faefa0b6abfcf56e8103096192",
    "jquery-migrate": "appleboy/jquery-migrate#1.2.1",
    "html5shiv": "aFarkas/html5shiv#3.7.2",
    "ladda": "hakimel/Ladda#0.9.3",
    "modernizr": "Modernizr/Modernizr#v2.8.1"
  },

npm ERR! Error: Problems were encountered
npm ERR! Please correct and try again.
npm ERR! extraneous: [email protected] /Users/kud/Projects/_playmedia/desktop/node_modules/html5shiv
npm ERR! extraneous: [email protected] /Users/kud/Projects/_playmedia/desktop/node_modules/hint
npm ERR! extraneous: [email protected] /Users/kud/Projects/_playmedia/desktop/node_modules/ladda
npm ERR! invalid: [email protected] /Users/kud/Projects/_playmedia/desktop/node_modules/ladda
npm ERR! missing: spin.js@^2.0.0, required by [email protected]
npm ERR!     at shrinkwrap_ (/usr/local/lib/node_modules/npm/lib/shrinkwrap.js:30:15)
npm ERR!     at /usr/local/lib/node_modules/npm/lib/shrinkwrap.js:24:5
npm ERR!     at /usr/local/lib/node_modules/npm/lib/ls.js:44:30
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-installed/read-installed.js:130:5
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-installed/read-installed.js:258:14
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/read-installed/node_modules/slide/lib/async-map.js:48:11)
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-installed/read-installed.js:258:14
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/read-installed/node_modules/slide/lib/async-map.js:48:11)
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-installed/read-installed.js:258:14
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/read-installed/node_modules/slide/lib/async-map.js:48:11)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/npm/npm/issues>

npm ERR! System Darwin 13.1.0
npm ERR! command "node" "/usr/local/bin/npm" "shrinkwrap"
npm ERR! cwd /Users/kud/Projects/_playmedia/desktop
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.10
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/kud/Projects/_playmedia/desktop/npm-debug.log
npm ERR! not ok code 0

[shama]

Yes it is not compatible. It could be a bit more friendly if #18 is implemented but not sure if it would completely fix it.

[kud]

As first step, couldn't we just make napa ignored from npm-shrinkwrap?

[shama]

How would we do that?

[kud]

I have no bloody idea lol :)

I should have a look. It's quite interesting. ;)

[thanpolas]

got bit by this too, shrinkwrap is a requirement at the current stage / size of the project I cannot afford to not have it...

Looking for a workaround, any suggestions would be greatly appreciated.

[thanpolas]

So since we have a no-go from NPM i guess the next sensible solution would be to have our own conf file napa.json... I'm down with that... @shama any other play here?

(sorry for pressing, this is a pain in my production installs)

[kud]

napa.json would be the answer in my opinion.

[shama]

I'm looking into a way to get it to play nice with shrinkwrap. Hopefully we can get away with just adding a command napa shrinkwrap. I really don't want to create our own config file as that would make napa an official: yet another package manager™

[kud]

You're right about another package manager.

[thanpolas]

Yes indeed, even if we create a napa.json file that won't stop shrinkwrap from breaking as it'll find napa's extraneous packages in node_modules

[bjmiller]

Does the solution to #22 also have an effect on this?

If so, I think updating that key with the package names becomes a pretty important feature.

[shama]

@bjmiller Unfortunately it doesn't help with this issue as packages are reported extraneous even when put into bundledDependencies.

[troywarr]

I just ran into this myself. Any chance that a fix has been implemented or a workaround discovered?

[thanpolas]

Soooooo... @shama what's the plan?

[caseyWebb]

The bad part? It pretty much changes the way napa works entirely.
The good part? It also solves #22, and will make #31 trivial to implement.

So, here's how it goes...

• clone repos into napa_modules
• run npm --save-optional file:napa_modules/module-name

Saving it to optionalDependencies makes npm shrinkwrap and npm prune look the other way. Also, since repos aren't in node_modules, we don't have to delete the .git subdir anymore (ref #53), so as far as versioning goes, we can just run a git checkout REV, assuming HEAD if not supplied, in each napa_modules repo each time napa is ran. Thus ensuring the versions in package.json stay in sync w/ installed napa packages, and that when unspecified, the latest version will always be used.

UPDATE: Turns out, we wouldn't have to separate them into a napa_modules dir at all. We just have to add the package to optionalDependencies and ensure that the version numbers match.

i.e. clone into node_modules/module-name, create the package.json, and add "module-name": "1.0.0" to package.json -> optionalDependencies

This method fixes #22 as well.

[caseyWebb]

Also, the decision to use optionalDependencies vs dependencies is agnostic of which of those approaches is taken and is based on the fact that if we modified the dependencies field and that was then committed, npm would try to install a package that doesn't exist yet.

[Spriz]

Any news on this particular issue?

[thanpolas]

upgrade to node 4.x+ and you won't be needing napa any more

[linkabi9]

I am trying to implement shrinkwrap. Can someone help me understand what thanpolas is talking about. I tried googling for what he is talking about but his comment did not provide enough information to be useful. I would appreciate it. Thanks!

[caseyWebb]

I can't say what @thanpolas meant, but I can give a pessimistic word of warning. I had made pretty decent progress getting npm shrinkwrap to work, only to have an upgrade to npm stop me in my tracks; I've since abandoned it.

How are you going about implementing it?

[linkabi9]

since i assume that a napa dependency is inherently more volatile than an npm dependency, i didn't bother trying to get napa to play nice with npm shrinkwrap. Instead i added support for a "directory" option in napa-config. Since I can now house my napa dependencies external to node_modules (napa_modules), I'm able to shrinkwrap npm dependencies.

You can see the feature in my fork which needs tests written, so i haven't done a pull request.

https://github.com/linkabi9/napa#additional-configuration

[caseyWebb]

Good thinking. When used with webpack you could add napa_modules to resolve.modulesDirectories to require as if it lived in node_modules. I'm sure a similar option exists for Browserify and the like.

[thanpolas]

@linkabi9 with node 4+ and onwards you can directly link github repos using npm thus rendering napa obsolete.

[caseyWebb]

@thanpolas You could do that before I believe; the issue comes into play when there is no package.json. Unless I'm mistaken.

[linkabi9]

@thanpolas Are you referring to npm's ability to reference github repos in the shorthand (linkabi9/napa for example)? If so, the solution does not fulfill my requirements. napa allows me to include dependencies that are source controlled when they don't have a package.json. can you provide content?

*edit: just saw we said the same thing @caseyWebb

[linkabi9]

@caseyWebb i did the same thing to my webpack config. nice to know others think along the same lines. also happy that webpack supported this unique requirement with minimal effort.