Host a static Jekyll website (WIP)

[chrisbarless]

Resolves #1

So far just showing how I would go about creating this node.

As you can see there are official/popular (1MM+ implementations) Docker images for almost any project. I use docker-compose to orchestrate instances of the images and get them to talk to each other over Docker networks.

I've added an nginx reverse proxy, let'sencrypt nginx companion, and used them to serve a simple Jekyll fileserver over SSL here with very little config. docker-compose up does all the magic on any machine with Docker installed.

I also added the most popular CJDNS image to the docker-compose.yml file to show what that looks like, although it's not configured. I haven't dove into IPFS yet but here are the most popular IPFS projects https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=ipfs&starCount=0

Next steps will be to configure Jekyll to serve our actual site, and configure ipfs/cjdns setup

[benhylau]

@pierrebeaucamp hey I feel that this is your area of expertise, would appreciate if you can have a look

[benhylau]

Thanks for putting this together @chrisbarless !

[benhylau]

What email shall we use for this? Shall we make one for tools? https://github.com/tomeshnet/documents/blob/master/service_setup/email.md#public-accounts

[chrisbarless]

yes

[benhylau]

I guess we need to put github.com/tomeshnet/tomesh.net as source and add the webhook?

[chrisbarless]

something like that!

[benhylau]

We should probably run a particular tag than let it version float.

[pierrebeaucamp]

Thanks for pinging me @benhylau. In general, there doesn't seem to be anything wrong with this PR. If it gets the job done and you have someone supporting this architecture, I'd say go with it. But I want to give some more general comments on this approach:

• I don't know how you're currently running software on your servers. If you already provision it with Docker or Docker compose, this setup is fine, but if you don't, I'd consider keeping things consistent instead of adding the overhead of Docker and/or Docker compose.

• The nginx-proxy container mounts /var/run/docker.sock as a volume. This is necessary in order for it to discover other services and route traffic automatically. However this also means that this disables any isolation you can have between containers. I would treat this container as if it has root access to all the other containers on your system. The question comes down to if you trust the author(s) of that docker image. Given that it is pretty popular, it seems fine, but I would still keep an eye on it.

• Similar, while we're talking about paranoid security, the letsencrypt-nginx-proxy-companion has the same access level and manages your SSL certs for you. Again, seems like a trustworthy service, but it's worth keeping an eye on it.

• The jekyll container exposes too many ports - it seem that you only need port 4000 enabled.

• I would also recommend against running jekyll serve to host content. Sure, it's pretty practical that it builds your site and then also serves it, but I would try to separate the "building" from the "serving" portion of the system. Not only would this protect the original source files from accidental exposure through bugs, but you're also much better of serving files from a battle tested system like nginx instead of exploiting a developer option from jekyll.

• Lastly, I don't see the point of the cjdns container - it seems to do nothing at all, but I might be wrong.

Again, there is no right or wrong approach to devops. If this setup works, and more importantly, someone understands it and can maintain it, go with it. But given the requirements to just serve a static site behind SSL I would probably go with something like this:

• Use a third party CI system to build your website (i.e. travisCI or CircleCI). Push the results to your server, never the source code itself.

• Use a proper webserver to serve that content. You don't need a reverse proxy for it at all. Nginx + Letsencrypt can be a bit tricky to set up sometimes, so if you want to have something simple, I can recommend https://caddyserver.com (for disclosure, I did some minor contributions to this project).

With this approach, you don't need Docker compose at all and only have a single process running. You also don't need to trust multiple third parties, only a single one.

The downsite to all this is that it might not be as extensible as Docker compose, if you want to add things in the future. I also don't know about CJDNS. Again, my main recommendation is to pick one devops / deploy tool and stick with it. If you don't have anything set up on the server yet, you might as well go with docker compose.

Also as a general note: I'd always go with a separate CI system and try to build my own containers once the setup reaches a certain complexity (like this one).

Hope that helps. I don't follow this project too closely anymore, so I hope I didn't upset anyone by jumping in here.

[chrisbarless]

that's some really excellent advice @pierrebeaucamp, i will take it to heart. lots to think about

[benhylau]

@pierrebeaucamp thanks for the comments. To answer some of that:

I don't know how you're currently running software on your servers

Manually deployed on a Droplet, with step-by-step docs. The goal is to use provisioning tools and have those steps as code, reducing the manual steps.

Re: trusting container images, is there general best practices for this such as pinning particular versions of it so at least we are shielded from the "new malicious version" risk?

I would also recommend against running jekyll serve to host content

I agree. Can we jekyll generate static assets on the same rules as we do now (on new commit and daily), into a static folder? @garrying how is it currently done, do we just jekyll serve?

Lastly, I don't see the point of the cjdns container - it seems to do nothing at all, but I might be wrong.

We do need it for next phase of this project because we want to serve the site over cjdns, and add the monitoring system which requires pinging cjdns nodes to check if they are active.

The downsite to all this is that it might not be as extensible as Docker compose, if you want to add things in the future.

Yes we plan to add ipfs and dat daemons so the site can be served over those protocols as well. Then there is cjdns, prometheus server, etc. so it does sound like multiple docker containers would provide the right separation for manageability, although not necessarily the security boundaries the way we have it.

As @chrisbarless mentioned, your comments is definitely very much appreciated!

[garrying]

There's a bash script that does jekyll build and moves the static files to the www serving folder. Yeah, script runs daily and on push to github (I think we'll want to revise how we use webhook events since the node project we're using is not longer maintained)

I'll take a look at this PR on the weekend.

[dcwalk]

could crib back from our networks and set up a travis CI flow for web-avail website, but that seems like not the right flow for nodes