Commit Graph

204 Commits

Author SHA1 Message Date
James Houlahan 8c2096e813 Other: Bump liteapi to fix update merging algorithm 2022-11-20 12:10:48 +01:00
James Houlahan 2972e1273f GODT-2045: Timeouts should be considered network issues 2022-11-20 10:07:33 +00:00
James Houlahan eb2423b0ed Other: Move sending logic to smtp.go 2022-11-18 17:05:20 +01:00
James Houlahan 319d51cb80 Other: Bump gluon version to prevent crash on log failure 2022-11-17 12:52:23 +01:00
James Houlahan 31fb878bbd GODT-2070: Implement SASL login for SMTP
go-smtp now comes with out of the box support for SASL PLAIN but it
still requires manual implementation of SASL LOGIN (deprecated).
2022-11-16 13:48:31 +01:00
James Houlahan 59278913ca GODT-2037: Handle and log API refresh event 2022-11-16 13:48:31 +01:00
James Houlahan 2023df3ef8 Other: Log mailbox message counts at startup 2022-11-16 13:48:31 +01:00
James Houlahan 112d79c2be Other: Bump gluon version to include rebuilt parser libs 2022-11-16 13:48:31 +01:00
Leander Beernaert 8b0024d53e Other: Update Gluon to latest version 2022-11-16 13:48:30 +01:00
James Houlahan 847d6de6bf Other: Bump liteapi version 2022-11-16 13:48:30 +01:00
James Houlahan da41398340 Other: Bump liteapi to mitigate race condition 2022-11-16 13:48:30 +01:00
Jakub 1e29a5210f GODT-1954: Draft message support
Add special case handling for draft messages so that if a Draft is
updated via an event it is correctly updated on the IMAP client via a
the new `imap.MessageUpdated event`.

This patch also updates Gluon to the latest version.
2022-11-16 13:48:30 +01:00
James Houlahan c548ba85fe Other: Add more extensive logging 2022-11-16 13:48:30 +01:00
James Houlahan 75b788b793 GODT-1993: Use more efficient filtering for message deletion 2022-11-16 13:48:30 +01:00
James Houlahan 99745ac067 Other: Switch to faster message IDs route
The new API route lets us query exactly which message IDs a user has,
allowing us to begin syncing much faster than before.
2022-11-16 13:48:30 +01:00
James Houlahan a213b48f93 GODT-2002: Poll after SMTP send
After sending, a client might append to the sent folder over IMAP.
In this case, we perform deduplication and return the message ID of the
sent message. However, if we haven't already processed this message in
gluon, it doesn't work as expected.

This change polls the event stream immediately after send. Note that it
doesn't wait for these events to be processed; that should be done in a
follow-up commit.
2022-11-16 13:48:30 +01:00
James Houlahan 075e1ef236 Other: Fix wrong encoding used for public key during sending
The newer liteapi contact code saves keys as *crypto.Key, however the
legacy code expects them to be strings. There was a bug in connecting
the new code to the legacy code: it was assumed these strings were meant
to be a base64 encoded string but they were actually just raw string
bytes.
2022-11-16 13:48:30 +01:00
Leander Beernaert 29dcd5450f Other: Add FEATURE_TEST_LOG_IMAP env variable
When set will log IMAP commands during feature tests.

This patch also updates Gluon to the latest version.
2022-11-16 13:48:30 +01:00
James Houlahan 93d9ae32fc Other: Only listen on localhost and add send test
We should only listen on constants.Host when serving IMAP and SMTP.
This change fixes that. It also adds a test that we can send over SMTP
and receive over IMAP.
2022-11-16 13:48:30 +01:00
James Houlahan cb42358e2f Other: Bump gluon version to not set internal ID header twice 2022-11-16 13:48:30 +01:00
Leander Beernaert 8f420d728c GODT-1984: Handle permanent message deletion
Only delete messages when unlabeled from trash/spam if they only exists
in All Mail and (spam or trash).

This patch also ports delete_from_trash.feature and use status rather
than fetch to count messages in a mailboxes.
2022-11-16 13:48:30 +01:00
James Houlahan d066e32719 GODT-1986: Handle case where an address has no decryption entities
It's possible (but very rare, I don't think proton still allows it)
for an address to have no keys. If we try to load the address keyring
for such an address, this change logs a warning that no decryption
entities were found in the unlocked keyring.

It bumps liteapi to a version that does not return an error when no
keys could be unlocked.
2022-11-16 13:48:30 +01:00
James Houlahan 784896434d Other: Add vault editor CI tool 2022-11-16 13:48:30 +01:00
James Houlahan d376b88cf0 Other: Use msgpack instead of json in vault
msgpack is much faster at serializing and deserializing than json. Using
it in the vault gives a performance boost.
2022-11-16 13:48:30 +01:00
James Houlahan 2f40b030ec Other: Bump liteapi to v0.36.1 2022-11-16 13:48:30 +01:00
James Houlahan 036a416a25 GODT-1777: Message de-duplication in IMAP (+ cleanup) 2022-11-16 13:48:30 +01:00
Leander Beernaert 8e34b51c77 GODT-1940: Fix message encryption
Update liteapi to v0.36.0 to include message encryption fix and fix
compile errors related to update.
2022-11-16 13:48:30 +01:00
James Houlahan 1919610793 Other: Bump gluon version 2022-11-16 13:48:30 +01:00
James Houlahan 6fbf6d90dc Other: Fix IMAP/SMTP/Login leaks/race conditions
Depending on the timing of bridge closure, it was possible for the 
IMAP/SMTP servers to not have started serving yet. By grouping this in
a cancelable goroutine group (*xsync.Group), we mitigate this issue.

Further, depending on internet disconnection timing during user login,
it was possible for a user to be improperly logged in. This change 
fixes this and adds test coverage for it.

Lastly, depending on timing, certain background tasks (updates check,
connectivity ping) could be improperly started or closed. This change
groups them in the *xsync.Group as well to be closed properly.
2022-11-16 13:48:30 +01:00
James Houlahan 828385b049 Other: Fix user sync leaks/race conditions
This fixes various race conditions and leaks related to the user's sync
and API event stream. It was possible for a sync/stream to begin after a
user was already closed; this change prevents that by managing the
goroutines related to sync/stream within cancellable groups.
2022-11-16 13:48:30 +01:00
Leander Beernaert 6bbaf03f1f Other: Fix goroutine leaks in sync tests
Add missing Close calls.

Properly handle nil channel for `user.startSync`.

This patch also updated liteapi and Gluon to latest master and dev
version respectively.
2022-11-16 13:48:30 +01:00
James Houlahan 0f125196a6 Other: Bump go-smtp version to fix race condition
There was a race condition internal to the go-smtp library.
In order to fix it, a version bump was necessary.
However, this significantly changed the library interface.
2022-11-16 13:48:30 +01:00
James Houlahan 974735d415 Other: Bump liteapi version to fix goroutine leaks 2022-11-16 13:48:30 +01:00
James Houlahan 472b96795f Other: Bump liteapi version 2022-11-16 13:48:30 +01:00
James Houlahan 81f4ef609b Other: Mitigate double-unlock of user keyring
We need to unlock the user keyring anyway to unlock the address keyring,
so we should just return it instead of re-unlocking the user keyring
when sending a message.
2022-11-16 13:48:30 +01:00
James Houlahan c4343e0124 Other: Bump liteapi and clean up tests a bit 2022-11-16 13:48:30 +01:00
James Houlahan 04b6571cb8 Other: Handle Seen/Flagged IMAP flags when APPENDing a message
When an IMAP client appends a message to a mailbox, it can specify
which flags it wants the appended message to have. We need to handle
these in a proton-specific way; not-seen messages need to be imported
with the Unread bool set to true, and flagged messages need to
additionally be imported with the Starred label.
2022-11-16 13:48:30 +01:00
Leander Beernaert ca132881f9 Other: Update Gluon to v0.13.0 2022-11-16 12:26:09 +01:00
Leander Beernaert e0ff30e9a8 Other: Update gluon to v0.12.0 2022-11-16 12:26:09 +01:00
James Houlahan 023e7b2d32 Other: Bump liteapi version to v0.34.0 2022-11-16 12:26:09 +01:00
James Houlahan ef2dea89b4 Other: Safer vault 2022-11-16 12:26:09 +01:00
James Houlahan 593d86f3a7 Other: Single instance 2022-11-16 12:26:09 +01:00
James Houlahan fd63611b41 Other: Safer user types 2022-11-16 12:26:09 +01:00
James Houlahan d330000c8d Other: Put back old 2FA/two-pass flow in GUI 2022-11-16 12:26:09 +01:00
James Houlahan 1c922ca083 Other: Fix flaky cookies test 2022-11-16 12:26:09 +01:00
James Houlahan 14a578f319 Other: Linter fixes after bumping linter version 2022-11-16 12:26:09 +01:00
James Houlahan 03e14154a6 Other: Bump gluon version 2022-11-16 12:26:09 +01:00
James Houlahan 509a767e50 GODT-1657: More stable sync, with some tests 2022-11-16 12:26:09 +01:00
James Houlahan 9d69a2e565 GODT-1657: Stable sync (still needs more tests) 2022-11-16 12:26:09 +01:00
James Houlahan 39b366ee69 GODT-1650: Password as bytes in API login 2022-11-16 12:26:09 +01:00
James Houlahan edd326efd9 GODT-1650: Mixed case and failure sending tests 2022-11-16 12:26:09 +01:00
James Houlahan cce372fc50 GODT-1650: Test of end-to-end send with attachments (internal) 2022-11-16 12:26:09 +01:00
James Houlahan 0badd69409 GODT-1650: sending tests with attachments 2022-11-16 12:26:09 +01:00
James Houlahan c953b8030a GODT-1650: text/html sending tests 2022-11-16 12:26:09 +01:00
James Houlahan ba9368426c GODT-1650: Send extras 2022-11-16 12:26:09 +01:00
James Houlahan 6ac68984f2 GODT-1609: Fix bridge password encoding 2022-11-16 12:26:09 +01:00
Romain LE JEUNE b536b8707e GODT-1609: use byte array for password 2022-11-16 12:26:09 +01:00
James Houlahan 4fc22e25ba GODT-1650: Bump gluon to latest version 2022-11-16 12:26:09 +01:00
James Houlahan 0f9a9a377b GODT-1650: Bump liteapi to v0.31.1 2022-11-16 12:26:09 +01:00
James Houlahan e9672e6bba GODT-1815: Combined/Split mode 2022-11-16 12:26:08 +01:00
James Houlahan 1da1188351 GODT-1815: Upgrade to liteapi v0.31.0 2022-11-16 12:26:08 +01:00
James Houlahan 39433fe707 GODT-1779: Remove go-imap 2022-11-16 12:26:08 +01:00
Romain LE JEUNE 31514c8e31 GODT-1645: add a make target for gobinsec check 2022-10-26 06:31:40 +00:00
Romain LE JEUNE 96878e2247
GODT-1853: upgrade dependencies (including x/crypto) 2022-09-23 09:20:14 +02:00
Jakub 61c787b1c7 GODT-1833: Updage go.mod/sum. 2022-09-01 13:44:01 +02:00
James Houlahan 6bbe2d0e00 Other(refactor): Remove bridgeWrap from frontend interface 2022-09-01 13:31:10 +02:00
Xavier Michelon 0a1f349901 Other: require go 1.18 and update to golangci-lint to latest revision + fixes. 2022-09-01 13:23:07 +02:00
Xavier Michelon 72708d6e2c GODT-1667: bridge-gui spawns bridge process. [skip-ci]
Other: renaming of bridge-gui.
WIP: locate bridge exe.
WIP: bridge process launch.
WIP: cleaner closure of bridge.
WIP: grpcClient connection retries.
WIP: clean exit when bridge process is killed.

Fixed issues from MR review. [skip-ci].

WIP: Fixed gRPC case in CMakelists.txt [skip-ci]

It caused issues on Debian.

WIP: update gRPC/protobuf and tweaked CMakeLists.txt. [skip-ci]

WIP: Fixed a bug where splash screen could not be dismissed. [skip-ci]
2022-09-01 13:21:31 +02:00
Xavier Michelon c11fe3e1ab GODT-1554 / 1555: Implement gRPC go service and Qt 5 frontend C++ app.
WIP: updates

WIP: cache on disk and autostart.

WIP: mail, keychain and more.

WIP: updated grpc version in go mod file.

WIP: user list.

WIP: RPC service placeholder

WIP: test C++ RPC client skeleton.

Other: missing license script update.

WIP: use Qt test framework.

WIP: test for app and login calls.

WIP: test for update & cache on disk calls.

WIP: tests for mail settings calls.

WIP: all client tests.

WIP: linter fixes.

WIP: fix missing license link.

WIP: update dependency_license script for gRPC and protobuf.

WIP: removed unused file.

WIP: app & login event streaming tests.

WIP: update event stream tests.

WIP: completed event streaming tests.

GODT-1554: qt C++ frontend skeleton.

WIP: C++ backend declaration.

wip: started drafting user model.

WIP: users. not functional.

WIP: invokable methods

WIP: Exception class + backend 'injection' into QML.

WIP: switch to VCPKG to ease multi-arch compilation,  C++ RPC client skeleton.

WIP: Renaming and reorganisation

WIP:introduced new 'grpc' go frontend.

WIP: Worker & Oveerseer for thread management.

WIP: added log to C++ app.

WIP: event stream architecture on Go side.

WIP: event parsing and streamer stopping.

WIP: Moved grpc to frontend subfolder + use vcpkg for gRPC and protobuf.

WIP: windows building ok

WIP: wired a few messages

WIP: more wiring.

WIP: Fixed imports after rebase on top of devel.

WIP: wired some bool and string properties.

WIP: more properties.

WIP: wired cache on disk stuff

WIP: connect event watcher.

WIP: login

WIP: fix showSplashScreen

WIP: Wired login calls.

WIP: user list.

WIP: Refactored main().

WIP: User retrieval .

WIP: no shared pointer in user model.

WIP: fixed user count.

WIP: cached goos.

WIP: Wired autostart

WIP: beta channel toggle wired.

WIP: User removal

WIP: wired theme

WIP: implemented configure apple mail.

WIP: split mode.

WIP: fixed user updates.

WIP: fixed Quit from tray icon

WIP: wired CurrentEmailClient

WIP: wired UseSSLForSMTP

WIP: wired change ports .

WIP: wired DoH. .

WIP: wired keychain calls.

WIP: wired autoupdate option.

WIP: QML Backend clean-up.

WIP: cleanup.

WIP: moved user related files in subfolder. .

WIP: User are managed using smart pointers.

WIP: cleanup.

WIP: more cleanup.

WIP: mail events forwarding

WIP: code inspection tweaks from CLion.

WIP: moved QML, cleanup, and missing copyright notices.

WIP: Backend is not QMLBackend.

Other: fixed issues reported by Leander. [skip ci]
2022-09-01 13:21:29 +02:00
Xavier Michelon a4e54f063d GODT-1553: RPC definition and mocks
WIP: updates

WIP: cache on disk and autostart.

WIP: mail, keychain and more.

WIP: updated grpc version in go mod file.

WIP: user list.

WIP: RPC service placeholder

WIP: test C++ RPC client skeleton.

Other: missing license script update.

WIP: use Qt test framework.

WIP: test for app and login calls.

WIP: test for update & cache on disk calls.

WIP: tests for mail settings calls.

WIP: all client tests.

WIP: linter fixes.

WIP: fix missing license link.

WIP: update dependency_license script for gRPC and protobuf.

WIP: removed unused file.

WIP: app & login event streaming tests.

WIP: update event stream tests.

WIP: completed event streaming tests.
2022-09-01 13:20:56 +02:00
Leander Beernaert 136d514cf7 GODT-1626: Update gopenpgp v2.4.7
This patch also replaces the deprecated calls to `SeparateKeyAndData`
with `SplitMessage`.
2022-08-16 16:05:42 +02:00
Leander Beernaert 6e48345d54 GODT-1627: Update go-srvp to v0.0.5 2022-08-16 16:05:42 +02:00
Jakub b8f27cc7d2 GODT-1550: Update dependencies 2022-05-16 09:23:16 +00:00
Jakub c32a106898 GODT-1525: Add keybase/go-keychain/secretservice as new keychain helper. 2022-04-06 11:49:27 +00:00
Jakub 6671dd38ea GODT-1524: Logout issues with macOS. 2022-03-23 09:50:17 +01:00
Jakub f9f4ce996d GODT-1375: Add host architecture to sentry reports 2022-01-18 09:52:57 +00:00
Jakub c2ffae3799 GODT-1494: Update GopenPGP to 2.4.1 2022-01-14 10:20:51 +01:00
Jakub e9c05c5a6b GODT-1490: Update go sentry and openpgp 2022-01-10 15:48:46 +01:00
Jakub e0d07d67a0 GODT-22: Frontend-backend
- GODT-1246 Implement settings view.
- GODT-1257 GODT-1246: Account and Help view
- GODT-1298: Minimal working build (panics)
- GODT-1298: loading QML (needs Popup window)
- GODT-1298: WARN: Adding PopupWindow not possible!
    In therecipe qt the `quickwidgets` classes are within `quick` module, but
    forgot to add library and include paths into cgo flags. Therefore
    compilation fails and it would be hard to patch therecipe in order to
    fix it.

    I am not sure if rewrite PopupWindow into go would make any difference,
    therefore I decided to use normal QML Window without borders.
- GODT-1298: Rework status window, add backend props, slots and signals.
- GODT-1298: Users
- GODT-1298: Login
- GODT-1298: WIP Help and bug report
- GODT-1178: MacOS dock icon control
- GODT-1298: Help, bug report, update and events
- GODT-1298: Apple Mail config and Settings (without cache on disk)
2021-11-30 10:12:36 +01:00
James Houlahan 6bd0739013 GODT-1158: Store full messages bodies on disk
- GODT-1158: simple on-disk cache in store
- GODT-1158: better member naming in event loop
- GODT-1158: create on-disk cache during bridge setup
- GODT-1158: better job options
- GODT-1158: rename GetLiteral to GetRFC822
- GODT-1158: rename events -> currentEvents
- GODT-1158: unlock cache per-user
- GODT-1158: clean up cache after logout
- GODT-1158: randomized encrypted cache passphrase
- GODT-1158: Opt out of on-disk cache in settings
- GODT-1158: free space in cache
- GODT-1158: make tests compile
- GODT-1158: optional compression
- GODT-1158: cache custom location
- GODT-1158: basic capacity checker
- GODT-1158: cache free space config
- GODT-1158: only unlock cache if pmapi client is unlocked as well
- GODT-1158: simple background sync worker
- GODT-1158: set size/bodystructure when caching message
- GODT-1158: limit store db update blocking with semaphore
- GODT-1158: dumb 10-semaphore
- GODT-1158: properly handle delete; remove bad bodystructure handling
- GODT-1158: hacky fix for caching after logout... baaaaad
- GODT-1158: cache worker
- GODT-1158: compute body structure lazily
- GODT-1158: cache size in store
- GODT-1158: notify cacher when adding to store
- GODT-1158: 15 second store cache watcher
- GODT-1158: enable cacher
- GODT-1158: better cache worker starting/stopping
- GODT-1158: limit cacher to less concurrency than disk cache
- GODT-1158: message builder prio + pchan pkg
- GODT-1158: fix pchan, use in message builder
- GODT-1158: no sem in cacher (rely on message builder prio)
- GODT-1158: raise priority of existing jobs when requested
- GODT-1158: pending messages in on-disk cache
- GODT-1158: WIP just a note about deleting messages from disk cache
- GODT-1158: pending wait when trying to write
- GODT-1158: pending.add to return bool
- GODT-1225: Headers in bodystructure are stored as bytes.
- GODT-1158: fixing header caching
- GODT-1158: don't cache in background
- GODT-1158: all concurrency set in settings
- GODT-1158: worker pools inside message builder
- GODT-1158: fix linter issues
- GODT-1158: remove completed builds from builder
- GODT-1158: remove builder pool
- GODT-1158: cacher defer job done properly
- GODT-1158: fix linter
- GODT-1299: Continue with bodystructure build if deserialization failed
- GODT-1324: Delete messages from the cache when they are deleted on the server
- GODT-1158: refactor cache tests
- GODT-1158: move builder to app/bridge
- GODT-1306: Migrate cache on disk when location is changed (and delete when disabled)
2021-11-30 10:12:36 +01:00
Jakub 9d405a1549 GODT-1397: Update bbolt to v1.3.6 2021-11-09 10:01:08 +00:00
Jakub a219ecf3cb Other: fix go.sum 2021-10-21 13:25:44 +02:00
Jakub 8f1ca00c9d GODT-1318 go-srp version identifier in go.sum underlying commit is identical 2021-09-30 20:19:07 +02:00
James Houlahan b5d63783f7 GODT-1318: Bump gopenpgp to 2.2.2, go-srp to a843a0b9, go-crypto to 52430bf6 2021-09-17 11:54:55 +02:00
Jakub e0113ec267 GODT-219: Update to godog v0.12.1 2021-09-10 13:38:05 +02:00
James Houlahan 2cce29e858 GODT-1201: bump gopenpgp to 2.1.10 and update crypto time 2021-06-17 05:32:17 +00:00
Jakub fb98a797ba Merge branch 'release/james' into devel 2021-06-15 15:56:31 +02:00
Jakub 6be31bdeb2 Other go.mod 2021-06-15 15:55:02 +02:00
James Houlahan 5bf359d34f GODT-1193: don't use message.Read; permit non-UTF-8 charsets 2021-06-11 06:43:21 +00:00
Jakub a2029002c4 GODT-1155 Update gopenpgp and use go-srp 2021-05-27 16:43:44 +02:00
James Houlahan c69239ca16 Other: bump go-rfc5322 dependency to v0.8.0 2021-05-27 11:03:49 +02:00
James Houlahan 3dadad5131 GODT-1161: Guarantee order of responses when creating new message 2021-05-17 17:54:28 +02:00
Michal Horejsek 8109831c07 GODT-35: Finish all details and make tests pass 2021-04-30 05:41:39 +02:00
James Houlahan 2284e9ede1 GODT-35: New pmapi client and manager using resty 2021-04-30 05:34:36 +02:00
Jakub 7a15ebbd54 Other: update go.mod with qt docs 2021-04-27 17:11:23 +02:00
James Houlahan 50550d42b4 GODT-213: Message Builder 2021-04-15 09:51:08 +00:00
James Houlahan c0a8877018 Other: include latest go.mod/go.sum changes 2021-03-01 17:48:22 +01:00
Michal Horejsek 4e531d4524 GODT-1036 Event loop Sentry reporting of failures and refresh 2021-02-17 09:17:19 +00:00
Michal Horejsek 06b3ed9b85 GODT-317 Fix wrong total mailbox size in Apple Mail 2021-02-11 07:29:28 +00:00
Jakub c6107dbd4b GODT-919 GODT-1022 GODT-947 Logs and signals
+ Added startup logs
+ Added wait group for update notifications
+ Changed hooks in debug and trace level
2021-02-10 08:51:14 +00:00