End to end encryption in Go with Pusher Channels
You will need Go 1.8+ set up on your machine.
Privacy is a hot topic this days. Who has access to what and who can read my conversation with a friend. Pusher Channels offers three kinds of channels:
- Public
- Private
- Encrypted
Basically, all three perform the same functions - flexible pub/sub messaging and tons of others. But there are few differences between them. Public channels do not require client-server authentication in order to subscribe to events. Private channels take it a step further by requiring client-server authentication. Encrypted channels build on top of private channels by introducing security in the form of encrypted data.
Kindly take a look at the images above and spot the difference. Seen any yet ? In the first image which shows the Debug console for a public channel, you can see the data being sent to Pusher Channels contains some fields - title, content and createdAt. Now take a look at the second image, you will notice those fields are no longer present but instead you have a bunch of non-human readable content your application obviously didn’t create. The field called ciphertext is what the data you sent to Pusher Channels was converted to. The word ciphertext outside this discourse refers to encrypted and/or garbled data.
Understanding encrypted channels
As depicted above, an advantage of an encrypted channel is the ability to send messages only the server SDK and any of your connected clients can read. No one else - including Pusher - will be able to read the messages.
Remember that a client has to go through the authentication process too.
Pusher Channels uses one of the current top encryption algorithms available and that is Secretbox. On the server side, the application author is meant to provide an encryption key to be used for the data encryption. This encryption key never gets to Pusher servers, which is why you are the only one that can read messages in an encrypted channel.
But a question. If the encryption key never gets to Pusher servers, how is a connected client able to subscribe to an event in an encrypted channel and read/decrypt the message ? The answer resides in the authentication process. During authentication, a shared secret key is generated based off the master encryption key and the channel name. The generated shared secret key will be used to encrypt the data before being offloaded to Pusher Channels. The shared secret is also sent as part of a successful authentication response as the client SDK will need to store it as it will be used for decrypting encrypted messages it receives. Again notice that since the encryption key never leaves your server, there is no way Pusher or any other person can read the messages if they don’t go through the authentication process - which is going to be done by the client side SDK.
Note that this shared secret is channel specific. For each channel subscribed to, a new shared secret is generated.
Here is a sample response:
{
"auth": "3b65aa197f334949f0ef:ffd3094d43e1bb21d5eb849c3debcbba0f7dd32bddeb0bb7dd8441516029853d",
"channel_data": {
"user_id": "10",
"user_info": {
"random": "random"
}
},
"shared_secret": "oB4frIyBUiYVzbUSBFCBl7U5BxzW8ni6wIrO4UaYIeo="
}
Apart from privacy and security, another benefit encrypted channels provide is message authenticity and protection against forgery. So there is maximum guarantee that whatever message is being received was published by someone who has access to the encryption key.
Implementing encrypted channels
To show encrypted channels in practice, we will build a live feed application. The application will consist of a server and client. The server will be written in Go.
Before getting started, it will be nice to be aware of some limitations imposed by an encrypted channel. They are:
- Channel name(s) must begin with
private-encrypted-. Examples includeprivate-encrypted-dashboardorprivate-encrypted-grocery-list. If you provide an encryption key but fail to follow the naming scheme, your data will not be encrypted. - Client events cannot be triggered
- Channel and event names are not encrypted. This is for good reasons as events need to be dispatched to right clients and making sure an event in the Pusher Channels namespace -
pusher:- cannot be used.
Before proceeding, you will need to create a new directory called pusher-encrypted-feeds. Make sure to create it within your $GOPATH. It can be done by issuing the following command in a terminal:
$ mkdir pusher-encrypted-feeds
Prerequisites
- Golang
>=1.8 - A Pusher account
- OpenSSL tool.
If you are a Windows user, please note that you can make use of Git Bash since it comes with the OpenSSL toolkit.
Building the server
The first thing to do is to create a Pusher Channels account if you don’t have one already. You will need to take note of your app keys and secret as we will be using them later on in the tutorial.
In the pusher-encrypted-feeds directory, you will need to create another directory called server.
The next step of action is to create a .env file to contain the secret and key gotten from the dashboard. You should paste in the following contents:
// pusher-encrypted-feeds/server/.env
PUSHER_APP_ID="PUSHER_APP_ID"
PUSHER_APP_KEY="PUSHER_APP_KEY"
PUSHER_APP_SECRET="PUSHER_APP_SECRET"
PUSHER_APP_CLUSTER="PUSHER_APP_CLUSTER"
PUSHER_APP_SECURE="1"
PUSHER_CHANNELS_ENCRYPTION_KEY="PUSHER_CHANNELS_ENCRYPTION_KEY"
PUSHER_CHANNELS_ENCRYPTION_KEY will be the master encryption key used to generate the shared secret and it should be difficult to guess. It is also required to be a 32 byte encryption key. You can generate a suitable encryption key with the following command:
$ openssl rand -base64 24
You will also need to install some dependencies - the Pusher Go SDK and another for parsing the .env file you previously created. You can grab those dependencies by running:
$ go get github.com/joho/godotenv
$ go get github.com/pusher/pusher-http-go
You will need to create a main.go file and paste in the following content:
// pusher-encrypted-feeds/server/main.go
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"strings"
"sync"
"time"
"github.com/joho/godotenv"
pusher "github.com/pusher/pusher-http-go"
)
func main() {
port := flag.Int("http.port", 1400, "Port to run HTTP service on")
flag.Parse()
err := godotenv.Load()
if err != nil {
log.Fatal("Error loading .env file")
}
appID := os.Getenv("PUSHER_APP_ID")
appKey := os.Getenv("PUSHER_APP_KEY")
appSecret := os.Getenv("PUSHER_APP_SECRET")
appCluster := os.Getenv("PUSHER_APP_CLUSTER")
appIsSecure := os.Getenv("PUSHER_APP_SECURE")
var isSecure bool
if appIsSecure == "1" {
isSecure = true
}
client := &pusher.Client{
AppId: appID,
Key: appKey,
Secret: appSecret,
Cluster: appCluster,
Secure: isSecure,
EncryptionMasterKey: os.Getenv("PUSHER_CHANNELS_ENCRYPTION_KEY"),
}
mux := http.NewServeMux()
f := &feed{
mu: &sync.RWMutex{},
data: make(map[string]string, 0),
}
mux.Handle("/feed", createFeedTitle(client, f))
mux.Handle("/pusher/auth", authenticateUsers(client))
log.Println("Starting HTTP server")
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", *port), mux))
}
type feed struct {
data map[string]string
mu *sync.RWMutex
}
func (f *feed) exists(title string) bool {
f.mu.RLock()
defer f.mu.RUnlock()
_, ok := f.data[title]
return ok
}
func (f *feed) Add(title, content string) error {
if f.exists(title) {
return errors.New("title already exists")
}
f.mu.Lock()
defer f.mu.Unlock()
f.data[title] = content
return nil
}
const (
successMsg = "success"
errorMsg = "error"
)
func createFeedTitle(client *pusher.Client, f *feed) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
if r.Method == http.MethodOptions {
return
}
writer := json.NewEncoder(w)
type respose struct {
Message string `json:"message"`
Status string `json:"status"`
Timestamp int64 `json:"timestamp"`
}
if r.Method != http.MethodPost {
w.WriteHeader(http.StatusMethodNotAllowed)
writer.Encode(&respose{
Message: http.StatusText(http.StatusMethodNotAllowed),
Status: errorMsg,
Timestamp: time.Now().Unix(),
})
return
}
var request struct {
Title string `json:"title"`
Content string `json:"content"`
}
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
w.WriteHeader(http.StatusBadRequest)
writer.Encode(&respose{
Message: "Invalid request body",
Status: errorMsg,
Timestamp: time.Now().Unix(),
})
return
}
if len(strings.TrimSpace(request.Title)) == 0 {
w.WriteHeader(http.StatusBadRequest)
writer.Encode(&respose{
Message: "Title field is empty",
Status: errorMsg,
Timestamp: time.Now().Unix(),
})
return
}
if len(strings.TrimSpace(request.Content)) == 0 {
w.WriteHeader(http.StatusBadRequest)
writer.Encode(&respose{
Message: "Content field is empty",
Status: errorMsg,
Timestamp: time.Now().Unix(),
})
return
}
if err := f.Add(request.Title, request.Content); err != nil {
w.WriteHeader(http.StatusAlreadyReported)
writer.Encode(&respose{
Message: err.Error(),
Status: errorMsg,
Timestamp: time.Now().Unix(),
})
return
}
go func() {
_, err := client.Trigger("private-encrypted-feeds", "items", map[string]string{
"title": request.Title,
"content": request.Content,
"createdAt": time.Now().String(),
})
if err != nil {
fmt.Println(err)
}
}()
w.WriteHeader(http.StatusOK)
writer.Encode(&respose{
Message: "Feed item was successfully added",
Status: errorMsg,
Timestamp: time.Now().Unix(),
})
}
}
func authenticateUsers(client *pusher.Client) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// Handle CORS
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
if r.Method == http.MethodOptions {
return
}
params, err := ioutil.ReadAll(r.Body)
if err != nil {
w.WriteHeader(http.StatusBadRequest)
return
}
presenceData := pusher.MemberData{
UserId: "10",
UserInfo: map[string]string{
"random": "random",
},
}
response, err := client.AuthenticatePresenceChannel(params, presenceData)
if err != nil {
w.WriteHeader(http.StatusBadRequest)
return
}
w.Write(response)
}
}
In the above, we create an HTTP server with two endpoints:
/pusher/authfor authentication of client SDKs./feedfor the addition of a new feed item.
Note that the feed items will not be stored in a persistent database but in memory instead
You should be able to run the server now. That can be done with:
$ go run main.go
Building the client
The client is going to contain three pages:
- a dashboard page
- a form page for adding new feed items
- a feed page for displaying feed items in realtime as received from the encrypted channel.
You will need to create a directory called client. That can be done with:
$ mkdir client
To get started, we will need to build the form page to allow new items to be added. You will need to create a file called new.html with:
$ touch new.html
In the newly created new.html file, paste the following content:
<!-- pusher-encrypted-feeds/client/new.html -->
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Pusher realtime feed</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="icon" type="image/x-icon" href="favicon.ico" />
<base href="/" />
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.min.css">
<style>
.hidden { display: none }
</style>
<body>
<section class="section">
<div class="container">
<div class="columns">
<div class="column is-5">
<h3 class="notification">Create a new post</h3>
<div class="notification is-success hidden" id="success"></div>
<div class="is-danger notification hidden" id="error"></div>
<form id="feed-form">
<div class="field">
<label class="label">Title : </label>
<div class="control">
<input
class="input"
type="text"
placeholder="Post title"
name="title"
id="title"
/>
</div>
</div>
<div><label>Message: </label></div>
<div>
<textarea
rows="10"
cols="70"
name="content"
id="content"
></textarea>
</div>
<button id="submit" class="button is-info">
Send
</button>
</form>
</div>
<div class="is-7"></div>
</section>
</body>
<script src="app.js"></script>
</html>
This is as simple as can be. We reference the Bulma css library, we create a form with an input and text field. Finally we link to a non-existent file called app.js - we will create that in a bit.
To view what this file looks like, you should navigate to the client directory and run the following command:
$ python -m http.server 8000
Here I used Python’s inbuilt server but you are free to use whatever.
You should visit localhost:8000/new.html . You should be presented with something similar to the image below:
As said earlier, we linked to a non-existent file app.js, we will need to create it and fill it with some code. Create the app.js file with:
$ touch app.js
In the newly created file, paste the following:
// pusher-encrypted-channels/client/app.js
(function() {
const submitFeedBtn = document.getElementById('feed-form');
const isDangerDiv = document.getElementById('error');
const isSuccessDiv = document.getElementById('success');
if (submitFeedBtn !== null) {
submitFeedBtn.addEventListener('submit', function(e) {
isDangerDiv.classList.add('hidden');
isSuccessDiv.classList.add('hidden');
e.preventDefault();
const title = document.getElementById('title');
const content = document.getElementById('content');
if (title.value.length === 0) {
isDangerDiv.classList.remove('hidden');
isDangerDiv.innerHTML = 'Title field is required';
return;
}
if (content.value.length === 0) {
isDangerDiv.classList.remove('hidden');
isDangerDiv.innerHTML = 'Content field is required';
return;
}
fetch('http://localhost:1400/feed', {
method: 'POST',
body: JSON.stringify({ title: title.value, content: content.value }),
headers: {
'Content-Type': 'application/json',
},
}).then(
function(response) {
if (response.status === 200) {
isSuccessDiv.innerHTML = 'Feed item was successfully added';
isSuccessDiv.classList.remove('hidden');
setTimeout(function() {
isSuccessDiv.classList.add('hidden');
}, 1000);
return;
}
if (response.status === 208) {
message = 'Feed item already exists';
} else {
message = response.statusText;
}
isDangerDiv.innerHTML = message;
isDangerDiv.classList.remove('hidden');
},
function(error) {
isDangerDiv.innerHTML = 'Could not create feed item';
isDangerDiv.classList.remove('hidden');
}
);
});
}
})();
In the above, we validate the form whenever the Send button is clicked. If the form contains valid data, it is sent to the Go server for processing. The server will store it and trigger a message to Pusher Channels.
Go ahead and submit the form. If successful and you are on the Debug Console, you will notice something of the following sort:
The next point of action will be to create the feeds page so entries can be viewed in realtime. You will need to create a file called feed.html. That can be done with:
$ touch feed.html
In the new file, paste the following HTML code:
<!-- pusher-encrypted-channels/client/feed.html -->
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Pusher realtime feed</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="icon" type="image/x-icon" href="favicon.ico" />
<base href="/" />
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.min.css">
<body>
<section class="section">
<div class="container">
<h1 class="notification is-info">Your feed</h1>
<div class="columns">
<div class="column is-7">
<div id="feed">
</div>
</div>
</div>
</div>
</section>
</body>
<script src="https://js.pusher.com/4.3/pusher.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/handlebars.js/4.1.0/handlebars.min.js"></script>
<script src="app.js"></script>
</html>
This page is basically empty. It will be updated by the Channels client SDK as it receives data.
We are linking to the Pusher Channels client SDK and Handlebars. Handlebars is used to compile templates we will inject into the page.
To be able to receive and update the feeds page with data the app.js file has to be updated to make use of Pusher Channels. In app.js , append the following code:
// pusher-encrypted-feed/client/app.js
// Sample template to be injected
const tmpl = `
<div class="box">
<article class="media">
<div class="media-left">
<figure class="image is-64x64">
<img src="https://bulma.io/images/placeholders/128x128.png" alt="Image" />
</figure>
</div>
<div class="media-content">
<div class="content">
<p>
<strong>{{title}}</strong>
<small>{{createdAt}}</small> <br />
{{content}}
</p>
</div>
</div>
</article>
</div>
`;
const APP_KEY = 'PUSHER_APP_KEY';
const APP_CLUSTER = 'PUSHER_CLUSTER';
Pusher.logToConsole = true;
const pusher = new Pusher(APP_KEY, {
cluster: APP_CLUSTER,
authEndpoint: 'http://localhost:1400/pusher/auth',
});
const channel = pusher.subscribe('private-encrypted-feeds');
// Use Handlebars to compile the template
const template = Handlebars.compile(tmpl);
const feedDiv = document.getElementById('feed');
channel.bind('items', function(data) {
// replace some fields in the template with data from the event.
const html = template(data);
const divElement = document.createElement('div');
divElement.innerHTML = html;
// Update the page
feedDiv.appendChild(divElement);
});
Remember to replace both
PUSHER_CLUSTERandPUSHER_KEYwith your credentials
With the addition above, the entire app.js should look like:
// pusher-encrypted-feed/client/app.js
(function() {
const submitFeedBtn = document.getElementById('feed-form');
const isDangerDiv = document.getElementById('error');
const isSuccessDiv = document.getElementById('success');
if (submitFeedBtn !== null) {
submitFeedBtn.addEventListener('submit', function(e) {
isDangerDiv.classList.add('hidden');
isSuccessDiv.classList.add('hidden');
e.preventDefault();
const title = document.getElementById('title');
const content = document.getElementById('content');
if (title.value.length === 0) {
isDangerDiv.classList.remove('hidden');
isDangerDiv.innerHTML = 'Title field is required';
return;
}
if (content.value.length === 0) {
isDangerDiv.classList.remove('hidden');
isDangerDiv.innerHTML = 'Content field is required';
return;
}
fetch('http://localhost:1400/feed', {
method: 'POST',
body: JSON.stringify({ title: title.value, content: content.value }),
headers: {
'Content-Type': 'application/json',
},
}).then(
function(response) {
if (response.status === 200) {
isSuccessDiv.innerHTML = 'Feed item was successfully added';
isSuccessDiv.classList.remove('hidden');
setTimeout(function() {
isSuccessDiv.classList.add('hidden');
}, 1000);
return;
}
if (response.status === 208) {
message = 'Feed item already exists';
} else {
message = response.statusText;
}
isDangerDiv.innerHTML = message;
isDangerDiv.classList.remove('hidden');
},
function(error) {
isDangerDiv.innerHTML = 'Could not create feed item';
isDangerDiv.classList.remove('hidden');
}
);
});
}
const tmpl = `
<div class="box">
<article class="media">
<div class="media-left">
<figure class="image is-64x64">
<img src="https://bulma.io/images/placeholders/128x128.png" alt="Image" />
</figure>
</div>
<div class="media-content">
<div class="content">
<p>
<strong>{{title}}</strong>
<small>{{createdAt}}</small> <br />
{{content}}
</p>
</div>
</div>
</article>
</div>
`;
const APP_KEY = 'PUSHER_APP_KEY';
const APP_CLUSTER = 'PUSHER_CLUSTER';
Pusher.logToConsole = true;
const pusher = new Pusher(APP_KEY, {
cluster: APP_CLUSTER,
authEndpoint: 'http://localhost:1400/pusher/auth',
});
const channel = pusher.subscribe('private-encrypted-feeds');
const template = Handlebars.compile(tmpl);
const feedDiv = document.getElementById('feed');
channel.bind('items', function(data) {
const html = template(data);
const divElement = document.createElement('div');
divElement.innerHTML = html;
feedDiv.appendChild(divElement);
});
})();
You can go ahead to open the feed.html page on a tab and new.html in another. Watch closely as whatever data you submit in new.html appears in feed.html. You can also keep an eye on the Debug Console to make sure all data is encrypted.
To make this app a little more polished, add an index.html page. You can find the source code at the accompanying GitHub repository of this tutorial.
Conclusion
In this tutorial, I introduced you to a lesser known feature of Pusher Channels - end to end encryption with encrypted channels. We also built an application that uses encrypted channels instead of the regular public channels you might be used to.
As always, the entire code for this article can be found on GitHub.
6 March 2019
by Lanre Adelowo