4274e887c1
|
||
|---|---|---|
| .. | ||
| acceptance | ||
| assets | ||
| cli | ||
| cli-test | ||
| config | ||
| initdb.d | ||
| migrations | ||
| public | ||
| scripts | ||
| seeders | ||
| src | ||
| test | ||
| views | ||
| .dockerignore | ||
| .editorconfig | ||
| .eslintrc.json | ||
| .gitignore | ||
| .rspec | ||
| .sequelizerc | ||
| .yamllint | ||
| .yardopts | ||
| Dockerfile | ||
| Dockerfile.migrations | ||
| Makefile | ||
| README.adoc | ||
| docker-compose.yml | ||
| essentials.yaml | ||
| ingest-update-center | ||
| package-lock.json | ||
| package.json | ||
| prepare-essentials | ||
| wait-for-postgres.sh | ||
README.adoc
Evergreen Services
This directory contains the numerous microservices which work in concert to provide functionality for the Evergreen distribution system.
|
Note
|
This project uses a consistent version of Node for all tasks via a Docker
container. To pull Node CLI tools into your path for interactive use, execute
|
Dashboard
The developer Dashboard is the hub of activity for Jenkins developers to manage the release life-cycle for Evergreen applications.
Pusher
Pusher is responsible for maintaining Server-sent Event connections for all the present evergreen-client instances.
Status
The Status service is responsible for knowing the current status of Jenkins instances in the ecosystem.
This includes:
-
The
SHA1ofessentials.yamlinstalled on the instance. -
Versions of all, essential and other, plugins
-
When the Jenkins process was last listed as online
-
❓ Number of Pipelines.
-
Environment capabilities of the instance (AWS, Azure, Docker, Kubernetes, etc)
Data Model
This is still a work-in-progress for the data model to back the Status service. The goal for this data model is to provide an relatively accurate and up-to-date representation of the fleet.
The model below is obviously, and intentionally, relational. Most of the data
which Status requires is relational, and the service must manage various
transitions along those relationships. For example, when a new version of the
"Evergreen manifest" (essentials.yaml) is created, a new Update record
must be created for each Instance to relate to.
Non-relational data is, however, contained in the data model below within the
Capabilities and Version record. In both cases, it is expected that the
service will be receiving semi-arbitrary blobs of JSON from instances.
+------------------------+
| versions |
+------------------------+
| id |
+--+ *uuid |
| | manifest (JSON) |
+------------+ | | manifestSchemaVersion |
| instances | | | checksum (unique) |
+------------+ | | createdAt |
| id | | | updatedAt |
| *uuid <---++ +------------------------+
| timezone | ^^
| flavor | ||
+----------------+ +---+ updateId | ||
| updates | | | createdAt | ||
+----------------+ | | updatedAt | ||
| id <---+ +------------+ ||
| commit | ||
| channel | || +---------------------+
| manifest (JSON)| || | capabilities |
| tainted | || +---------------------+
| createdAt | || | id |
| updatedAt | |+--+ *uuid |
+----------------+ | | capabilities (JSON)|
| | createdAt |
| | updatedAt |
| +---------------------+
|
|
|
| +-----------------+
| | connections |
| +-----------------+
| | id |
+---+ *uuid |
| lastConnected |
| createdAt |
| updatedAt |
+-----------------+
* denotes an index
`id` fields are always considered primary keysVersions
The versions table should be considered to be much more of an audit trail of
versions rather than having a One-to-one relationship with an instance.
For the time being this means that a since instance will have multiple
records in versions, and the application must select the most recent
createdAt time in order to have an accurate picture of the instance.
Storage Considerations
In the model above it is expected that a single Channel record is going to
exist for each channel (alpha, canary, ga). Each change to essentials.yaml
should create a new Update record.
Where there is room for non-trivial data growth is with Instance,
Connection, Capabilities, and Version which should all have a single
record for each Jenkins Evergreen instance.
Models
Migrations for the database can be found in migrations/ and are based on the
Sequelize library.
In order to create a new Model, execute the following from this directory:
% source activate
% sequelize model:generate --helpAnd then follow the help text. The Sequelize Models and Migrations documentation has more information, as does the Variable/Data Types reference.
Telemetry
The Telemetry service is responsible for receiving error and user telemetry from Jenkins instances for processing in the Dashboard by developers.
This includes:
-
Exception/error reports
-
Uncaught exceptions
-
Logged errors
-
Logged warnings
-
Agent errors
-
-
❓
How to generate a log into Jenkins manually for testing
If you need to force a log to be sent to the backend, a possible hack is to open the /script URL, then paste and run the following code:
java.util.logging.Logger.getLogger('some.logger.name').warning('Warning log for testing');Environment Variables
| Name | Description |
|---|---|
|
Set with any value to turn on Sequelize database query tracing |
|
Set with the secret to use for creating JSON Web Tokens for authentication |
Debugging
The best way to debug the services is to use the Chrome Dev Tools in conjunction with Node’s debugger.
-
Create a unit test in the
test/directory which is going to exercise the functionality you wish to debug. -
Add a
debugger;statement into your code where you wish to create a breakpoint. -
Open Chrome and navigate to about://inspect
-
In the
services/directory execute:make debug-unit -
Select the appropriate Remote Target in the Chrome Dev Tools.
-
Start debugging!
Database Debugging
Debugging database queries can be done by enabling the
Sequelize
logging by setting the DB_TRACING environment variable, for example:
DB_TRACING=true make checkIn order to poke around in the development database being used for testing,
after running make migrate you can inspect the docker-compose db instance
with:
make debug-dbThis will open up the psql command line client for PostgreSQL and should
allow you to inspect and query the tables being used a development instance of
the backend services.