rtyler
/
brokenco.de
1
0
Fork 0
Code Pull Requests Releases Activity

Copy edits

This commit is contained in:
R Tyler Croy 2019-02-03 15:34:46 -08:00
parent 34ce477c7e
commit f10a7c3453
No known key found for this signature in database
GPG Key ID: E5C92681BEF6CEA2
2 changed files with 31 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Makefile
View File

@ -3,7 +3,7 @@ all:
LANG="en_US.UTF-8" jekyll build LANG="en_US.UTF-8" jekyll build
drafts: tags drafts: tags
LANG="en_US.UTF-8" jekyll serve --drafts LANG="en_US.UTF-8" jekyll serve --drafts --incremental
tags: tags:
./generate-tags ./generate-tags

58
_posts/2019-02-02-codevalet-the-failed-experiment.md
View File

@ -15,18 +15,19 @@ realized that I have never really reflected and shared what it was and more
importantly: why it failed. At it's core, Code Valet was intended to solve two importantly: why it failed. At it's core, Code Valet was intended to solve two
fundamental problems: firstly, I wanted a [Jenkins fundamental problems: firstly, I wanted a [Jenkins
Pipeline](https://jenkins.io/doc/book/pipeline/) as a Service, since I find Pipeline](https://jenkins.io/doc/book/pipeline/) as a Service, since I find
Jenkins [Declarative] Pipeline to be a very useful tool. Secondly, the Jenkins Jenkins Pipeline to be a very useful tool, which I wanted for _all_ my open
source projects. Secondly, the Jenkins
project needed to shift its footing towards a continuous feedback and project needed to shift its footing towards a continuous feedback and
continuous delivery model. Code Valet aimed to solve both of these problems. continuous delivery model. Code Valet aimed to solve both of these problems.
CloudBees had tried to run "Jenkins as a Service" with its now (finally!) Years ago, CloudBees had tried to run "Jenkins as a Service" with its now (finally!)
defunct "DEV@Cloud" service, or Kohsuke's "Buildhive" project. DEV@Cloud defunct "DEV@Cloud" service, or in a different manner with Kohsuke's "Buildhive" project. DEV@Cloud
provided users with a _full_ Jenkins installation, which I believe was part of provided users with a _full_ Jenkins installation, which I believe was part of
its undoing. The essential balancing act of developer tools is that we must its undoing. The essential balancing act of developer tools is that we must
provide our customers with enough power and flexibility, but not so much that provide our customers with enough power and flexibility, but not so much that
they shoot themselves in the foot. Providing a full Jenkins installation, to they shoot themselves in the foot. Providing a full Jenkins installation to
many people, is not only giving them the gun, but loading it, pointing it at many people is not only giving them the gun, but loading it, pointing it at
their foot, and then daring them not to shoot. Jenkins is **powerful** but their foot, and then daring them not to shoot. Jenkins is **powerful** but
unfortunately it can become unwieldy and a support nightmare. "Jenkins unfortunately it can become unwieldy and a support nightmare. "Jenkins
Pipeline as a Service" meant something different to me, Code Valet would allow Pipeline as a Service" meant something different to me, Code Valet would allow
@ -42,15 +43,15 @@ projects this was _perfect_, I could easily add a `Jenkinsfile` to my GitHub
repositories, and things just worked! repositories, and things just worked!
To address the second challenge, Code Valet's Jenkins image was **bleeding To address the second problem, Code Valet's Jenkins image was **bleeding
edge**. Not in the way [Jenkins edge**. Not in the way [Jenkins
Evergreen](https://jenkins.io/projects/evergreen) is bleeding edge, but more Evergreen](https://jenkins.io/projects/evergreen) is bleeding edge, but more
Gentoo Linux-style. Jenkins core was built daily from the `HEAD` of the master Gentoo Linux-style. Jenkins core was **built daily** from the `HEAD` of the master
branch. The batteries-included plugins were also **all** built from their branch. The batteries-included plugins were also **all built daily** from their
respective master branches. The Git plugin, Azure VM Agents plugin, Pipeline respective master branches. The Git plugin, Azure VM Agents plugin, Pipeline
plugins, _everything in the system was the absolute latest_. As you may have plugins, _everything in the system_ was the absolute latest. As you may have
guessed, this resulted in dozens of very interesting build and runtime failures guessed, this resulted in dozens of interesting build and runtime failures,
which I reported upstream. For the first time, I had an environment which was all of which were reported upstream. For the first time, I had an environment which was
providing "real world" SaaS-style feedback, using [Sentry](https://sentry.io) providing "real world" SaaS-style feedback, using [Sentry](https://sentry.io)
on _real_ Jenkins workloads. on _real_ Jenkins workloads.
@ -63,8 +64,8 @@ the CloudBees product roadmap.
### Lessons ### Lessons
For my taste, that bleeding edge turned out to be a little too For my taste, that bleeding edge turned out to be a little too
rough. I filed two dozen tickets for various plugins, some of which were bloody. I filed two dozen tickets for various plugins, some of which were
addressed very quickly, and a couple more which remain open. Rapid feedback is addressed very quickly, and others remained open for weeks, some still remain open. Rapid feedback is
only useful with rapid(ish) iteration. Like many large and federated open only useful with rapid(ish) iteration. Like many large and federated open
source communities, the Jenkins project suffers a bit from uneven levels of source communities, the Jenkins project suffers a bit from uneven levels of
investment across the plugin ecosystem. For example, while Microsoft's investment across the plugin ecosystem. For example, while Microsoft's
@ -79,7 +80,7 @@ Evergreen](https://github.com/jenkins-infra/evergreen).
Before I built Code Valet, I could have likely filled half a book with thoughts Before I built Code Valet, I could have likely filled half a book with thoughts
and practices on security practices for Jenkins. Originally designed with and practices on good security for Jenkins. Originally designed with
internal development teams in mind, running Jenkins on a hostile network like internal development teams in mind, running Jenkins on a hostile network like
the public internet is an uphill battle, updating default configuration the public internet is an uphill battle, updating default configuration
settings, disabling functionality to reduce the attack surface area, and settings, disabling functionality to reduce the attack surface area, and
@ -87,10 +88,10 @@ ensuring security best practices are being enforced and adhered to by the
users. In the case of Code Valet, everything got _far more challenging._. users. In the case of Code Valet, everything got _far more challenging._.
Combine the existing lock down procedures for a Jenkins on the public internet, Combine the existing lock down procedures for a Jenkins on the public internet,
and then add the requirement to lock down Jenkins far beyond what it natively and then add the requirement to lock down Jenkins far beyond what it natively
supports, in order to provide users with that limited "Jenkins Pipeline as a supports in order to provide users with that limited "Jenkins Pipeline as a
Service" experience. I ended up writing a significant amount of Groovy code to Service" experience. I ended up writing a significant amount of Groovy code to
configure and adjust settings throughout Jenkins, as [Configuration as configure and adjust settings throughout Jenkins, as [Configuration as
Code](https://jenkins.io/projects/jcasc/) didn't exist at the time. At times Code](https://jenkins.io/projects/jcasc/) didn't yet exist. At times
this still wasn't enough, and I resorted to clever container or nginx hacks to this still wasn't enough, and I resorted to clever container or nginx hacks to
disable, hide, or otherwise obfuscate certain aspects of Jenkins' behavior. disable, hide, or otherwise obfuscate certain aspects of Jenkins' behavior.
[All of these hacks](https://github.com/codevalet/master) I still consider to [All of these hacks](https://github.com/codevalet/master) I still consider to
@ -107,24 +108,24 @@ The cost model for Code Valet was never something I expected to be
net-positive. In discussions I would refer to it as a "loss leader", something net-positive. In discussions I would refer to it as a "loss leader", something
to drive adoption of Jenkins Pipeline with a calculated user acquisition cost. to drive adoption of Jenkins Pipeline with a calculated user acquisition cost.
Still, Code Valet was deployed onto Kubernetes (AKS) and very intentionally Still, Code Valet was deployed onto Kubernetes (AKS) and very intentionally
restricted to reduce per-user cost and overhead wherever possible. Tightly restricted to reduce overhead and per-user cost wherever possible. Tightly
packing Jenkins master containers into Kubernetes, and then dynamically packing Jenkins master containers into Kubernetes, and then dynamically
provisioning Azure VM and container agents for workloads remain design patterns provisioning Azure VM and container agents for workloads remain design patterns
I stand by for cheaply running Jenkins-as-a-Service, but Jenkins remains I stand by for cheaply running Jenkins-as-a-Service, but Jenkins is
undoubtedly **huge**. I could not get decent performance in a low enough memory undoubtedly **huge**. I could not get decent performance in a low enough memory
and CPU footprint for Code Valet _not_ to be a loss leader. And while Kohsuke's and CPU footprint for Code Valet _not_ to be a loss leader. And while Kohsuke's
Buildhive project was, if I recall, running one big multi-tenant Jenkins Buildhive project was, if I recall, running one big multi-tenant Jenkins
instance, Code Valet by design used per-user instances. The exact numbers I instance, Code Valet by design used per-user instances. The exact numbers I
cannot remember, but I spent my time thinking of more and more novel, yet cannot remember, but I spent my time thinking of more and more novel yet
non-invasive ways to reduce the monthly cost per user. Looking across the non-invasive ways to reduce the monthly cost per user. Looking across the
market today, it's very clear how punishing the race to the bottom for price on market today, it is evident how punishing the race to the bottom for price on
CI-as-a-Service products has been. Most vendors spend a non-trivial amount of CI-as-a-Service products has been. Most vendors spend a non-trivial amount of
time finding clever ways of engaging in AWS Spot Instance arbitrage or other time finding ingenious ways to arbitrage with AWS Spot Instances or other
means of shaving pennies of compute-hours where possible. All this before means of shaving pennies off compute-hours where possible. This was all very
GitHub Actions recently showed up on the scene and dropped the bottom out of competitive even before GitHub Actions showed up on the scene and dropped the
the market entirely. In my opinion, developer tools has always been a bottom out of the market entirely. In my opinion, developer tools has always
challenging market to work in. Developers inherently undervalue their tools, been a challenging market to work in. Developers inherently undervalue their
recognizing the importances of a $3000 laptop or a $1000 chair to daily tools, recognizing the importances of a $3000 laptop or a $1000 chair to daily
productivity, we still balk at $15/month services or licenses which would productivity, we still balk at $15/month services or licenses which would
otherwise improve our lives. otherwise improve our lives.
@ -132,8 +133,9 @@ otherwise improve our lives.
Early in 2018, CloudBees acquired [Code Ship](https://codeship.com), bringing Early in 2018, CloudBees acquired [Code Ship](https://codeship.com), bringing
onboard a great product and engineering team, but also a pretty good onboard a great product and engineering team, but also a pretty good
CI-as-a-Service offering. While Code Ship doesn't speak Jenkins Pipeline, it CI-as-a-Service offering. While Code Ship doesn't speak Jenkins Pipeline, it
was already quite mature with significant market traction, helping close the was already quite a mature product with significant market traction, helping
book on the Code Valet experiment. close the book on the Code Valet experiment.
--- ---