= RFC-0005: JSON over HTTP for data serialization and transport
:toc: preamble
:toclevels: 3
ifdef::env-github[]
:tip-caption: :bulb:
:note-caption: :information_source:
:important-caption: :heavy_exclamation_mark:
:caution-caption: :fire:
:warning-caption: :warning:
endif::[]

.**RFC Template**

.Metadata
[cols="1h,1"]
|===
| RFC
| 0005

| Title
| JSON over HTTP for data serialization and transport

| Sponsor
| link:https://github.com/rtyler/[R Tyler Croy]

| Status
| Draft :speech_balloon:

| Type
| Standards

| Created
| 2020-01-07

|===

== Abstract

Otto is a distributed system in which multiple services need to communicate
with one another in standardized formats with a common transport layer.
Practically every component in the system acts as both a producer and a
consumer of data. Consolidating all service-to-service interactions onto a
single format and transport layer offers numerous benefits in tooling and
shared code. It is also very important to make the Otto ecosystem easily
interoperable, such that users can bring their own services into the Otto
ecosystem without requiring any implementation changes from the existing
services.

== Specification

This specification describes the standard methoa and format for all
service-to-service interaction in the Otto distributed system. The interactions
in the system are expected to take two forms:

. *Synchronous*: a traditional request/response model with two services, or a
  client and service, directly interacting with one another. This flow is expected
  to follow the <<rest-api, REST API model>> described below.
. *Asynchronous*: oriented around the Otto eventbus, whereby services can send a
  message to a channel and expect no response other than a confirmation that the
  message was received. The details for event transmission and interaction with
  the Eventbus is described in the <<events, Events>> section below.

[[rest-api]]
=== REST API

Generally speaking, REST APIs are intended for administrative for
user-interface related data requirements. For the most part, services are
expected to communicate with one another using <<events, Events on the
Eventbus>> rather than sending synchronous requests back and forth.

All REST APIs must be:

* *Versioned*: The URLs should be versioned using the major version number from
  the service's
  link:http://semver.org/[Semantic Versioning]
  scheme. For example, `GET /v1/channels`. 
* *Documented with OpenAPI*: 
* *Accept JSON*: All requests to REST APIs must be documented JSON payloads,
  except in cases of binary object uploads
* *Respond with JSON*: All REST APIs are expected to _only_ respond with
  documented JSON payloads. Any non-JSON response, except for expected binary
  blobs, are to be considered by the client as a server-side error.

==== Error Handling

Errors should be communicated by the server with JSON encoded error messages
and status codes:

* *4xx*: Malformed request or other client errors. Clients should not attempt
  to retry these errors, but instead log them as local errors and try to
  recover.
* *5xx*: Server-side failures. Clients _should* attempt to retry the requests
  after an exponential back-off, with some permanent failure mode after a
  period of 30+ minutes.


If the server does *not* respond with a JSON payload, the client should treat
the response as if it is a recoverable error and perform an exponential
back-off retry.


[[events]]
=== Events


Services are all expected to emit and receive events over a
link:https://en.wikipedia.org/wiki/WebSocket[WebSocket]
connection to the Eventbus service, e.g. `http://eventbus.lan/ws/`. The Eventbus acts as a quasi-central broker
of all events occurring in the Otto distributed system. Its design is discussed
in another RFC< but the message payloads sent to "channels" on the eventbus
*must* be JSON data structures matching JSON Schemas defined for each channel.


==== Error Handling

For cases where the WebSocket connection has errored, reset, or is otherwise
unavailable, services must attempt to retry into infinity. Local error logs
should be generated during these failure schenarios and the service's health
check URLs should also convey that the Eventbus connection is down.

The only errors which should be expected by services writing events to the
Eventbus should be schema validation errors, which are considered to be similar
to REST API client-side errors  Therefor local errors should be logged, but the
message should not be retransmitted.


== Reasoning

Interprocess communication occurring via JSON over HTTP helps meet a number of
the key design goals for Otto:

* *Avoiding a singular central hub*
* *Extensibility must not come at the expense of system integrity*
* *User-level extension of the system must be viable*


The key aspect of JSON over HTTP which helps address all of these goals is that
JSON and HTTP are interoperable with practically _everything_ in the modern
technology environment, from backend services, to cloud storage providers, and
also user's browsers.

By providing documented synchronous and asynchronous APIs, Otto should be able
to accomodate new user-provided services into its ecosystem that add additional
functionality without any substantive change to the core components shipped
with Otto. One example which comes to mind is an analytics system around Otto.
In its current design there is not analytics system tracking tasks executed,
resources utilized, etc. If a user or provider wished to offer budget
forecasting, as an example, this JSON over HTTP model with REST APIs and events
via WebSockets would allow for the deployment of services which could tabulate
interesting events off of the Eventbus, and easily provide the end-user with a
forecast. Implementing such a system, assuming the necessary events are present
in the Eventbus and the necessary data is available from existing services'
REST APIs, would require *zero* changes to the "base" components of Otto.


Since browsers can also natively "speak" JSON over HTTP, this also allows for
new and interesting user interfaces atop these services and events to be
developed.

=== Alternatives Considered

==== Protocol Buffers / gRPC

The only serious alternative considered was the use of
link:https://developers.google.com/protocol-buffers/[Protocol Buffers]
for the data format/serialization and gRPC for the transport layer between services.

This approach would be functionally very similar, and may even offer some
schema and data validation benefits when compared to JSON over HTTP.

The gRPC approach does add implementation overhead however for new clients,
which must have shared binary stubs with other services generated from the
Protocol Buffer IDL footnote:[Interactive Data Language, in Protocol Buffers
the language for describing the binary data structure].
This increases the coupling between services, which must all be booted with
shared definitions of data, but also prevents easy interoperability with
"outside" services and web browsers.


Ultimately, the portability story for Protocol Buffers and gRPC pales in
comparison to JSON over HTTP. For Otto, the extensibility and portability of
data in the system is a key design goal.

== Security


The authentication and authorization of connections over HTTP is not subject to
this specification. Additional concerns around data verification through
signatures or other means are also not intended to be discussed in this
specification. The securing of the transport layer is a concern which will be
addressed in future extensions to this design.


== Testing

There are no testing issues specifically related to this proposal. However
clients and services will need to express API documentation which can be
automatically tested in the future. Tooling to help this already exists, for
example
link:https://github.com/apiaryio/dredd[Dredd]
which works with OpenAPI/Swagger specifications.


== References

* link:https://github.com/actix/actix-web[actix-web]: a Rust framework for
  building high-performance web services.
* link:https://docs.serde.rs/serde_json/[serde_json]: a Rust library for
  serializing and deserializing JSON objects
* link:https://json-schema.org/learn/getting-started-step-by-step.html[JSON Schema]: 
  tool for specifying JSON data structure and requirements.
* link:https://swagger.io/specification/[OpenAPI/Swagger specification]