I am building Haskell web services using docker. I do this utilising the stack tool within the official Haskell Docker image. Doing this has a number of advantages - It's predictable - I can control bringing in new versions of stack and ghc. - It's easy to share - no stack or Haskell installation is required to build the image - These services can interact with other services in a controlled environment using Docker compose. One downside is that I doing this naively means waiting for a modules to download every time docker build is run since the stack build command will start from scratch each time. I worked around this by adding two build steps. The first runs a hello world example with the stack.yml and package.yml from my project. The second build builds the intended code. This trick separated the downloading of dependent libraries from building the code and means day to day code changes are quick to build since the first stack build is cached as a docker image layer. Still though, making a small change to a stack yml file means a long wait re-downloading all dependent libraries. Firstly, is there a better way of doing this? I know stack supports building within docker but when I worked this way I still found projects depended on the system stack and would commonly fail due to stack bugs. Secondly, my two "stack build" calls where the first builds a hello world program feels a bit hacky. is there a better way to achieve this? Thanks, Dan
Dear Dan, On Tue, 2018-09-04 at 12:21 +0100, Daniel Rolls via Haskell-Cafe wrote:
Secondly, my two "stack build" calls where the first builds a hello world program feels a bit hacky. is there a better way to achieve this?
I cannot comment on the Stack parts of your questions, since I don't currently use it. Regarding the Docker parts, however, have you tried something like COPY [ \ "cabal.config", \ "*.cabal", \ "./"] RUN cabal update \ && \ cabal install -j --only-dependencies prior to your main compilation layer? That would install the dependencies once, with the layer cache expiring on change to the lib declarations, without the need for the 'hello world' program you describe. In case you don't freeze your dependencies, you can likely remove the `cabal.config` line. Peace, tiredpixel
We're dealing with this same hassle at Slant. For starters, instead of your hello-world build, you can use: stack build --dependencies-only We've gone a few steps farther to make Kubernetes deployment easier. We have a base container for web services with this Dockerfile: FROM haskell:8.2.1 as buildenv WORKDIR /app RUN apt-get update && apt-get -y install make xz-utils libpq-dev RUN stack upgrade --binary-version 1.6.1 RUN stack update && stack setup --resolver lts-11.22 RUN stack build Cabal --resolver lts-11.22 RUN stack build haskell-src-exts --resolver lts-11.22 RUN stack build lens --resolver lts-11.22 RUN stack build aeson --resolver lts-11.22 RUN stack build http-conduit --resolver lts-11.22 RUN stack build servant-server --resolver lts-11.22 We'll occasionally make a new version of this with a bumped LTS version. When we started there was no official 8.4-base docker image, but since we're using stack it doesn't really matter. The explicit install of stack-1.6.1 is due to an error that occurs when trying to pull indexes with 1.7.1 on the kubernetes build server; there's probably a cleaner solution but we're not using any 1.7.1 features yet. Optimally the order here should be "least likely to change goes first" but we haven't done any real analysis on that.
From here, individual services have a Dockerfile roughly like so:
FROM debian:jessie as runtimeenv RUN apt-get update && apt-get install -y libgmp-dev && rm -rf /var/lib/apt/lists/* FROM <private>/haskell-webserver:lts-11.22 as dependencies # pre-build any additional slow local dependencies COPY stack.yaml . COPY package.yaml . COPY submodules ./submodules RUN rm -rf ~/.stack/indices/ RUN stack build --dependencies-only FROM dependencies as build COPY . . RUN stack build --ghc-options="-O2" # additional options as desired FROM runtimeenv WORKDIR /app COPY --from=build /app/.stack-work/install/x86_64-linux/lts-11.22/8.2.2/bin/app-name-exe . CMD ./app-name-exe This gives us a reasonable build speed and nice lean deploy container. On Tue, Sep 4, 2018 at 4:22 AM Daniel Rolls via Haskell-Cafe < haskell-cafe@haskell.org> wrote:
I am building Haskell web services using docker. I do this utilising the stack tool within the official Haskell Docker image. Doing this has a number of advantages
- It's predictable - I can control bringing in new versions of stack and ghc. - It's easy to share - no stack or Haskell installation is required to build the image - These services can interact with other services in a controlled environment using Docker compose.
One downside is that I doing this naively means waiting for a modules to download every time docker build is run since the stack build command will start from scratch each time. I worked around this by adding two build steps. The first runs a hello world example with the stack.yml and package.yml from my project. The second build builds the intended code. This trick separated the downloading of dependent libraries from building the code and means day to day code changes are quick to build since the first stack build is cached as a docker image layer. Still though, making a small change to a stack yml file means a long wait re-downloading all dependent libraries.
Firstly, is there a better way of doing this? I know stack supports building within docker but when I worked this way I still found projects depended on the system stack and would commonly fail due to stack bugs.
Secondly, my two "stack build" calls where the first builds a hello world program feels a bit hacky. is there a better way to achieve this?
Thanks, Dan
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
participants (3)
-
Daniel Rolls -
Theodore Lief Gannon -
tiredpixel