ProtonMail
/
proton-bridge
mirror of https://github.com/ProtonMail/proton-bridge
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch master into devel

This commit is contained in:
Jakub 2020-07-02 10:16:08 +02:00
parent 2512d3647a 5348ae7d18
commit 35ae2011b6
12 changed files with 84 additions and 60 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

78
Changelog.md
View File

@ -4,51 +4,22 @@ Changelog [format](http://keepachangelog.com/en/1.0.0/)
## Unreleased
### Added
* GODT-360 Detect charset embedded in html/xml.
### Changed
* GODT-354 Do not label/unlabel messsages from `All Mail` folder.
* GODT-388 Support for both bridge and import/export credentials by package users.
* GODT-387 Store factory to make store optional.
* GODT-386 Renamed bridge to general users and keep bridge only for bridge stuff.
* GODT-308 Better user error message when request is canceled.
* GODT-312 Validate recipient emails in send before asking for their public keys.
* GODT-368 Bump docker-credential-helpers version.
* GODT-280 Migrate to gopenpgp v2:
* `Unlock()` call on pmapi-client unlocks both User keys and Address keys.
* Salt is available via `AuthSalt()` method.
* GODT-394 Don't check SMTP message send time in integration tests.
* GODT-280 Migrate to gopenpgp v2.
* `Unlock()` call on pmapi-client unlocks both User keys and Address keys.
* Salt is available via `AuthSalt()` method.
### Fixed
* GODT-356 Fix crash when removing account while mail client is fetching messages (regression from GODT-204).
* GODT-390 Don't logout user if AuthRefresh fails because internet was off.
* GODT-358 Bad timeouts with Alternative Routing.
* GODT-363 Drafts are not deleted when already created on webapp.
* GODT-390 Don't logout user if AuthRefresh fails because internet was off.
* GODT-341 Fixed flaky unittest for Store synchronization cooldown.
* Crash when failing to match necessary html element.
* Crash in message.combineParts when copying nil slice.
* Handle double charset better by using local ParseMediaType instead of mime.ParseMediaType.
* Don't remove log dir.
* GODT-422 Fix element not found (avoid listing credentials, prefer getting).
* GODT-404 Don't keep connections to proxy servers alive if user disables DoH.
* Ensure DoH is used at startup to load users for the initial auth.
* Issue causing deadlock when reloading users keys due to double-locking of a mutex.
* Correctly handle failure to unlock single key.
* GODT-479 Fix flaky integration tests.
* GODT-484 Fix infinite loop when decoding invalid 2231 charset
## [v1.2.7] Donghai-hotfix - beta (2020-05-07)
### Added
* IMAP mailbox info update when new mailbox is created.
* GODT-72 Use ISO-8859-1 encoding if charset is not specified and it isn't UTF-8.
### Changed
* GODT-360 Detect charset embedded in html/xml.
* GODT-354 Do not label/unlabel messsages from `All Mail` folder.
* GODT-388 Support for both bridge and import/export credentials by package users.
* GODT-387 Store factory to make store optional.
* GODT-386 Renamed bridge to general users and keep bridge only for bridge stuff.
* GODT-312 Validate recipient emails in send before asking for their public keys.
* GODT-368 Bump docker-credential-helpers version.
* GODT-394 Don't check SMTP message send time in integration tests.
* GODT-280 Migrate to gopenpgp v2.
* `Unlock()` call on pmapi-client unlocks both User keys and Address keys.
* Salt is available via `AuthSalt()` method.
* GODT-308 Better user error message when request is canceled.
* GODT-162 User Agent does not contain bridge version, only client in format `client name/client version (os)`.
* GODT-258 Update go-imap to v1.
@ -72,13 +43,38 @@ Changelog [format](http://keepachangelog.com/en/1.0.0/)
* Dead code from `pkg/message`.
### Fixed
* GODT-356 Fix crash when removing account while mail client is fetching messages (regression from GODT-204).
* GODT-358 Bad timeouts with Alternative Routing.
* GODT-363 Drafts are not deleted when already created on webapp.
* GODT-390 Don't logout user if AuthRefresh fails because internet was off.
* GODT-341 Fixed flaky unittest for Store synchronization cooldown.
* Crash when failing to match necessary html element.
* Crash in message.combineParts when copying nil slice.
* Handle double charset better by using local ParseMediaType instead of mime.ParseMediaType.
* Don't remove log dir.
* GODT-422 Fix element not found (avoid listing credentials, prefer getting).
* GODT-404 Don't keep connections to proxy servers alive if user disables DoH.
* Ensure DoH is used at startup to load users for the initial auth.
* Issue causing deadlock when reloading users keys due to double-locking of a mutex.
* Correctly handle failure to unlock single key.
* GODT-479 Fix flaky integration tests.
* GODT-484 Fix infinite loop when decoding invalid 2231 charset
* GODT-267 Correctly detect if a message is a draft even if does not have DraftLabel.
* GODT-308 Reduce minimum read speed threshold to avoid issues with flaky internet.
* GODT-321 Changing address ordering would cause all messages to disappear in combined mode.
* GODT-129 Fix custom message PGP by using template.
## [v1.2.7] Donghai-hotfix - beta (2020-05-07)
## [v1.2.8] Donghai-fix-append (beta 2020-06-XXX)
### Changed
* GODT-396 reduce number of EXISTS calls.
* GODT-143 Allow appending to Sent folder when sender matches account address.
### Fixed
* GODT-502 Fixed crash when unable to parse a message header.
## [v1.2.7] Donghai-fix-sync - (beta 2020-05-07 live 2020-04-20)
### Added
* IMAP extension MOVE with UIDPLUS support.

8
internal/bridge/release_notes.go
View File

@ -15,14 +15,12 @@
// You should have received a copy of the GNU General Public License
// along with ProtonMail Bridge. If not, see <https://www.gnu.org/licenses/>.
// Code generated by ./release-notes.sh at Tue 09 Jun 2020 10:21:50 AM CEST. DO NOT EDIT.
// Code generated by ./release-notes.sh at Thu 21 May 2020 07:59:59 AM CEST. DO NOT EDIT.
package bridge
const ReleaseNotes = `• Changed directory for storing logs on Linux
const ReleaseNotes = `
`
const ReleaseFixedBugs = `• Fixed sync errors due to localization
• Fixed draft deletion
• Fixed re-appearing messages in Outlook
const ReleaseFixedBugs = `• Fixed ignored import to Sent folder
`

4
internal/imap/mailbox_message.go
View File

@ -131,10 +131,6 @@ func (im *imapMailbox) CreateMessage(flags []string, date time.Time, body imap.L
// We didn't find the message in the store, so we are currently sending it.
logEntry.WithField("time", date).Info("No matching UID, continuing APPEND to Sent")
// For now we don't import user's own messages to Sent because GetUIDByHeader is not smart enough.
// This will be fixed in GODT-143.
return nil
}
// This is an APPEND to the Sent folder, so we will set the sent flag

5
internal/store/mailbox_ids.go
View File

@ -219,6 +219,11 @@ func (storeMailbox *Mailbox) GetUIDByHeader(header *mail.Header) (foundUID uint3
// in PM message. Message-Id in normal copy/move will be the PM internal ID.
messageID := header.Get("Message-Id")
// There is nothing to find, when no Message-Id given.
if messageID == "" {
return uint32(0)
}
// The most often situation is that message is APPENDed after it was sent so the
// Message-ID will be reflected by ExternalID in API message meta-data.
externalID := strings.Trim(messageID, "<> ") // remove '<>' to improve match

1
internal/store/mailbox_message.go
View File

@ -303,7 +303,6 @@ func (storeMailbox *Mailbox) txCreateOrUpdateMessages(tx *bolt.Tx, msgs []*pmapi
seqNum,
msg,
)
shouldSendMailboxUpdate = true
}
continue
}

2
internal/store/user_message_test.go
View File

@ -80,7 +80,7 @@ func TestCreateOrUpdateMessageMetadata(t *testing.T) {
a.Equal(t, []*pmapi.Attachment(nil), msg.Attachments)
a.Equal(t, int64(-1), msg.Size)
a.Equal(t, "", msg.MIMEType)
a.Equal(t, mail.Header(nil), msg.Header)
a.Equal(t, make(mail.Header), msg.Header)
// Change the calculated data.
wantSize := int64(42)

4
pkg/pmapi/events_test.go
View File

@ -20,6 +20,7 @@ package pmapi
import (
"fmt"
"net/http"
"net/mail"
"regexp"
"strconv"
"strings"
@ -113,6 +114,7 @@ var (
EventItem: EventItem{ID: "hdI7aIgUO1hFplCIcJHB0jShRVsAzS0AB75wGCaiNVeIHXLmaUnt4eJ8l7c7L6uk4g0ZdXhGWG5gfh6HHgAZnw==", Action: EventCreate},
Created: &Message{
ID: "hdI7aIgUO1hFplCIcJHB0jShRVsAzS0AB75wGCaiNVeIHXLmaUnt4eJ8l7c7L6uk4g0ZdXhGWG5gfh6HHgAZnw==",
Header: make(mail.Header),
Subject: "Hey there",
},
},
@ -153,6 +155,7 @@ var (
EventItem: EventItem{ID: "msgID1", Action: EventCreate},
Created: &Message{
ID: "id",
Header: make(mail.Header),
Subject: "Hey there",
},
},
@ -160,6 +163,7 @@ var (
EventItem: EventItem{ID: "msgID2", Action: EventCreate},
Created: &Message{
ID: "id",
Header: make(mail.Header),
Subject: "Hey there again",
},
},

10
pkg/pmapi/messages.go
View File

@ -24,6 +24,7 @@ import (
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/mail"
@ -32,6 +33,7 @@ import (
"strings"
"github.com/ProtonMail/gopenpgp/v2/crypto"
"github.com/sirupsen/logrus"
"golang.org/x/crypto/openpgp/packet"
)
@ -220,9 +222,13 @@ func (m *Message) UnmarshalJSON(b []byte) error {
if raw.Header != "" && raw.Header != "(No Header)" {
msg, err := mail.ReadMessage(strings.NewReader(raw.Header + "\r\n\r\n"))
if err == nil {
m.Header = msg.Header
if err != nil {
logrus.WithField("rawHeader", raw.Header).Trace("Failed to parse header")
return fmt.Errorf("failed to parse header of message %v: %v", m.ID, err.Error())
}
m.Header = msg.Header
} else {
m.Header = make(mail.Header)
}
return nil

4
release-notes/bugs.txt
View File

@ -1,3 +1 @@
• Fixed sync errors due to localization
• Fixed draft deletion
• Fixed re-appearing messages in Outlook
• Fixed ignored import to Sent folder

1
release-notes/notes.txt
View File

@ -1 +0,0 @@
• Changed directory for storing logs on Linux

12
test/README.md
View File

@ -36,6 +36,14 @@ graph LR
We want to test Bridge app from outside as much as possible. So we mock server (API),
credentials store and call commands to IMAP or SMTP the same way as client would do.
## Running tests
In order to run Integration tests just go into the test folder `cd test`
and run `make test`.
You can also test only specific feature (or subset of features) by using `FEATURES` environment
variable: `FEATURES=features/imap/message/create.feature make test`.
## Example test
BDD test in gherkin (cucumber) format (https://cucumber.io/docs/gherkin/reference/).
@ -99,6 +107,10 @@ we can always be sure what each steps does or should do.
In the code, we separate those parts in its own files to make sure
it's clear how the function should be implemented.
In the `Given` phase is also generally better to setup data (as `there are messages...`)
first, then users (`there is connected user...`) and then connections (`there is IMAP client...`).
This can prevent some hitches in internal implementation of integration tests.
## API faked by fakeapi or liveapi
We need to control what server returns. Instead of using raw JSONs,

15
test/features/imap/message/create.feature
View File

@ -18,7 +18,6 @@ Feature: IMAP create messages
| from | to | subject | read |
| [primary] | john.doe@email.com | foo | true |
@ignore
Scenario: Creates message sent from user's primary address
Given there is IMAP client selected in "Sent"
When IMAP client creates message "foo" from address "primary" of "userMoreAddresses" to "john.doe@email.com" with body "hello world" in "Sent"
@ -29,7 +28,6 @@ Feature: IMAP create messages
| [primary] | john.doe@email.com | foo | true |
And mailbox "INBOX" for "userMoreAddresses" has no messages
@ignore
Scenario: Creates message sent from user's secondary address
Given there is IMAP client selected in "Sent"
When IMAP client creates message "foo" from address "secondary" of "userMoreAddresses" to "john.doe@email.com" with body "hello world" in "Sent"
@ -57,3 +55,16 @@ Feature: IMAP create messages
| from | to | subject | read |
| notuser@gmail.com | alsonotuser@gmail.com | foo | true |
And mailbox "INBOX" for "userMoreAddresses" has no messages
# Importing duplicate messages when messageID cannot be found in Sent already.
#
# Previously, we discarded messages for which sender matches account address to
# avoid duplicates, but this led to discarding messages imported through mail client.
Scenario: Imports a similar (duplicate) message to sent
Given there are messages in mailbox "Sent" for "userMoreAddresses"
| from | to | subject | body |
| [primary] | chosen@one.com | Meet the Twins | Hello, Mr. Anderson |
And there is IMAP client selected in "Sent"
When IMAP client creates message "Meet the Twins" from address "primary" of "userMoreAddresses" to "chosen@one.com" with body "Hello, Mr. Anderson" in "Sent"
Then IMAP response is "OK"
And mailbox "Sent" for "userMoreAddresses" has 2 messages