In this article we are going to setup a caching layer in our Golang application by using Redis. REDIS is REmote DIctionary Server which is why we called Redis. Redis is an in-memory, key-value data store.
REDIS Overview
You can find more details via Introduction of REDIS , here is the overview
Install REDIS
You can install Redis on Linux, Windows, macOS . Redis is not officially supported on Windows. However, you can install Redis on Windows for development running on WSL2 (Windows Subsystem for Linux).
On Ubuntu, Add the repository to the apt index, update it, and then install:
curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
sudo apt-get update
sudo apt-get install redisLastly, start the Redis server like so:
$ sudo service redis-server start
Starting redis-server: redis-server.You can test that your Redis server is running by connecting with the Redis CLI:
$ redis-cli
127.0.0.1:6379> ping
PONGWorking with REDIS CLI
Read the page Redis CLI here for more details. We already the Redis server running here, so quickly check the port where is running
$ ps -ef | grep redis
redis 756 9 0 10:11 ? 00:00:00 /usr/bin/redis-server 127.0.0.1:6379
tvt 765 10 0 10:17 pts/0 00:00:00 grep --color=auto redisWe can see here Redis is running on localhost with default port 6379. Now let’s use Redis command line interface to executes some common operations, mostly the GET and SET operations. Note that Redis provide 16 databases out of the box, we can select the database that we are going to use by using the index number of database from 0 to 15.
$ redis-cli
127.0.0.1:6379> select 60
(error) ERR DB index is out of range
127.0.0.1:6379> select 10
OK
127.0.0.1:6379[10]> SET key1 value
OK
127.0.0.1:6379[10]> GET key1
"value"
127.0.0.1:6379[10]> APPEND key1 1
(integer) 6
127.0.0.1:6379[10]> GET key1
"value1"
127.0.0.1:6379[10]> SET key2 value2
OK
127.0.0.1:6379[10]> KEYS *
1) "key2"
2) "key1"
127.0.0.1:6379[10]> SET key3 value3 EX 10
OK
127.0.0.1:6379[10]> GET key3
"value3"
127.0.0.1:6379[10]> GET key3
"value3"
127.0.0.1:6379[10]> GET key3
(nil)
127.0.0.1:6379[10]>Create caching layer
We are going to create a new folder with name “cache”, and then we are going to create an interface for our caching layer in the file posts-cache.go
package cache
import "github.com/favtuts/golang-mux-api/entity"
type PostCache interface {
Set(key string, value *entity.Post)
Get(key string) *entity.Post
}
Then we are going to create the concrete implementation of this interface by using Redis, so we need to execute the command bellow to install the Redis library
$ go get -u github.com/go-redis/redis/v7
go: downloading github.com/go-redis/redis/v7 v7.4.1
go: added github.com/go-redis/redis/v7 v7.4.1Then we have the full Redis cache implementation as follow
package cache
import (
"encoding/json"
"time"
"github.com/favtuts/golang-mux-api/entity"
"github.com/go-redis/redis/v7"
)
type redisCache struct {
host string
db int
expires time.Duration
}
func NewRedisCache(host string, db int, exp time.Duration) PostCache {
return &redisCache{
host: host,
db: db,
expires: exp,
}
}
func (cache *redisCache) getClient() *redis.Client {
return redis.NewClient(&redis.Options{
Addr: cache.host,
Password: "",
DB: cache.db,
})
}
func (cache *redisCache) Set(key string, post *entity.Post) {
client := cache.getClient()
// serialize Post object to JSON
json, err := json.Marshal(post)
if err != nil {
panic(err)
}
client.Set(key, json, cache.expires*time.Second)
}
func (cache *redisCache) Get(key string) *entity.Post {
client := cache.getClient()
val, err := client.Get(key).Result()
if err != nil {
return nil
}
post := entity.Post{}
err = json.Unmarshal([]byte(val), &post)
if err != nil {
panic(err)
}
return &post
}
Applying caching layer
Now we are going to include the caching layer within the post controller, so we can store the posts into memory using Redis and that will help us to improve the performance of the API.
package controller
import (
"encoding/json"
"net/http"
"strconv"
"strings"
"github.com/favtuts/golang-mux-api/cache"
"github.com/favtuts/golang-mux-api/entity"
"github.com/favtuts/golang-mux-api/errors"
"github.com/favtuts/golang-mux-api/service"
)
type controller struct{}
var (
postService service.PostService
postCache cache.PostCache
)
type PostController interface {
GetPostByID(response http.ResponseWriter, request *http.Request)
GetPosts(response http.ResponseWriter, request *http.Request)
AddPost(response http.ResponseWriter, request *http.Request)
}
// More effiction way to use Dependency Injection
func NewPostController(service service.PostService, cache cache.PostCache) PostController {
postService = service
postCache = cache
return &controller{}
}
func (*controller) GetPosts(response http.ResponseWriter, request *http.Request) {
response.Header().Set("Content-type", "application/json")
posts, err := postService.FindAll()
if err != nil {
response.WriteHeader(http.StatusInternalServerError)
json.NewEncoder(response).Encode(errors.ServiceError{Message: "Error getting the posts"})
}
response.WriteHeader(http.StatusOK)
json.NewEncoder(response).Encode(posts)
}
func (*controller) GetPostByID(response http.ResponseWriter, request *http.Request) {
response.Header().Set("Content-Type", "application/json")
postID := strings.Split(request.URL.Path, "/")[2]
var post *entity.Post = postCache.Get(postID)
if post == nil {
post, err := postService.FindByID(postID)
if err != nil {
response.WriteHeader(http.StatusNotFound)
json.NewEncoder(response).Encode(errors.ServiceError{Message: "No posts found!"})
return
}
postCache.Set(postID, post)
response.WriteHeader(http.StatusOK)
json.NewEncoder(response).Encode(post)
} else {
response.WriteHeader(http.StatusOK)
json.NewEncoder(response).Encode(post)
}
}
func (*controller) AddPost(response http.ResponseWriter, request *http.Request) {
response.Header().Set("Content-type", "application/json")
var post entity.Post
err := json.NewDecoder(request.Body).Decode(&post)
if err != nil {
response.WriteHeader(http.StatusInternalServerError)
json.NewEncoder(response).Encode(errors.ServiceError{Message: "Error unmarshalling data"})
return
}
err1 := postService.Validate(&post)
if err1 != nil {
response.WriteHeader(http.StatusInternalServerError)
json.NewEncoder(response).Encode(errors.ServiceError{Message: err1.Error()})
return
}
result, err2 := postService.Create(&post)
if err2 != nil {
response.WriteHeader(http.StatusInternalServerError)
json.NewEncoder(response).Encode(errors.ServiceError{Message: "Error saving the post"})
return
}
postCache.Set(strconv.FormatInt(post.ID, 10), &post)
response.WriteHeader(http.StatusOK)
json.NewEncoder(response).Encode(result)
}
Also, we are going to make changes for the controller testing with caching layer:
package controller
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/favtuts/golang-mux-api/cache"
"github.com/favtuts/golang-mux-api/entity"
"github.com/favtuts/golang-mux-api/repository"
"github.com/favtuts/golang-mux-api/service"
"github.com/stretchr/testify/assert"
)
const (
ID int64 = 123
TITLE string = "Title 1"
TEXT string = "Text 1"
)
var (
postRepo repository.PostRepository = repository.NewSQLiteRepository()
postSrv service.PostService = service.NewPostService(postRepo)
postCh cache.PostCache = cache.NewRedisCache("localhost:6379", 0, 100)
postController PostController = NewPostController(postSrv, postCh)
)
func TestAddPost(t *testing.T) {
// Create a new HTTP POST request
var jsonReq = []byte(`{"title": "Title 1","text": "Text 1"}`)
req, _ := http.NewRequest("POST", "/posts", bytes.NewBuffer(jsonReq))
// Assign HTTP Handler function (controller AddPost function)
handler := http.HandlerFunc(postController.AddPost)
// Record HTTP Response (httptest)
response := httptest.NewRecorder()
// Dispatch the HTTP request
handler.ServeHTTP(response, req)
// Add Assertions on the HTTP Status code and the response
status := response.Code
if status != http.StatusOK {
t.Errorf("Handler returnd a wrong status code: got %v want %v", status, http.StatusOK)
}
// Decode the HTTP response
var post entity.Post
json.NewDecoder(io.Reader(response.Body)).Decode(&post)
// Assert HTTP response
assert.NotNil(t, post.ID)
assert.Equal(t, TITLE, post.Title)
assert.Equal(t, TEXT, post.Text)
// Clean up database
// cleanUp(&post)
tearDown(post.ID)
}
func TestGetPosts(t *testing.T) {
// Insert new post
setup()
// Create a GET HTTP request
req, _ := http.NewRequest("GET", "/posts", nil)
// Assign HTTP Handler function (controller GetPosts function)
handler := http.HandlerFunc(postController.GetPosts)
// Record HTTP Response (httptest)
response := httptest.NewRecorder()
// Dispatch the HTTP request
handler.ServeHTTP(response, req)
// Add Assertions on the HTTP Status code and the response
status := response.Code
if status != http.StatusOK {
t.Errorf("Handler returnd a wrong status code: got %v want %v", status, http.StatusOK)
}
// Decode the HTTP response
var posts []entity.Post
json.NewDecoder(io.Reader(response.Body)).Decode(&posts)
// Assert HTTP response
assert.NotNil(t, posts[0].ID)
assert.Equal(t, TITLE, posts[0].Title)
assert.Equal(t, TEXT, posts[0].Text)
// Clean up database
// cleanUp(&posts[0])
tearDown(ID)
}
func cleanUp(post *entity.Post) {
postRepo.Delete(post)
}
func setup() {
var post entity.Post = entity.Post{
ID: ID,
Title: TITLE,
Text: TEXT,
}
postRepo.Save(&post)
}
func tearDown(postID int64) {
var post entity.Post = entity.Post{
ID: postID,
}
postRepo.Delete(&post)
}
func TestGetPostByID(t *testing.T) {
// Insert new post
setup()
// Create new HTTP request
req, _ := http.NewRequest("GET", "/posts/123", nil)
// Assing HTTP Request handler Function (controller function)
handler := http.HandlerFunc(postController.GetPostByID)
// Record the HTTP Response
response := httptest.NewRecorder()
// Dispatch the HTTP Request
handler.ServeHTTP(response, req)
// Assert HTTP status
status := response.Code
if status != http.StatusOK {
t.Errorf("Handler returned wrong status code: got %v want %v",
status, http.StatusOK)
}
// Decode HTTP response
var post entity.Post
json.NewDecoder(io.Reader(response.Body)).Decode(&post)
// Assert HTTP response
assert.Equal(t, ID, post.ID)
assert.Equal(t, TITLE, post.Title)
assert.Equal(t, TEXT, post.Text)
// Cleanup database
tearDown(ID)
}
Testing caching layer
We are going to run unit testing for rest API, then we are going to use Redis CLI to verify the result
$ cd controller/
$ go test -v -run TestAddPost
$ redis-cli
> select 0
> KEYS *
1) "3460388482403070121"
> GET 3460388482403070121
"{\"id\":3460388482403070121,\"title\":\"Title 1\",\"text\":\"Text 1\"}"
$ go test -v -run TestGetPostByID
$ redis-cli
> select 0
> KEYS *
1) "3460388482403070121"
2) "123"
> GET 123
"{\"id\":123,\"title\":\"Title 1\",\"text\":\"Text 1\"}"Serving caching layer
Now we are going to change the file server.go where we define all the endpoints of the API
package main
import (
"os"
"github.com/favtuts/golang-mux-api/cache"
"github.com/favtuts/golang-mux-api/controller"
router "github.com/favtuts/golang-mux-api/http"
"github.com/favtuts/golang-mux-api/repository"
"github.com/favtuts/golang-mux-api/service"
)
var (
postRepository repository.PostRepository = repository.NewSQLiteRepository()
postService service.PostService = service.NewPostService(postRepository)
postCache cache.PostCache = cache.NewRedisCache("localhost:6379", 1, 10)
postController controller.PostController = controller.NewPostController(postService, postCache)
httpRouter router.Router = router.NewMuxRouter()
)
func main() {
httpRouter.GET("/posts", postController.GetPosts)
httpRouter.POST("/posts", postController.AddPost)
httpRouter.GET("/posts/{id}", postController.GetPostByID)
httpRouter.SERVE(":" + os.Getenv("PORT"))
}
Let’s go to the terminal and run the REST API server
$ export PORT=8000
$ go run *.go
Mux HTTP server running on port :8000Now we use the cURL commands to test our REST API
127.0.0.1:6379[1]> KEYS *
(empty array)
curl -v --location 'http://localhost:8000/posts' \
--header 'Content-Type: application/json' \
--data '{
"title": "Title 1",
"text": "Text 1"
}'
{"id":7515018121727868634,"title":"Title 1","text":"Text 1"}
127.0.0.1:6379[1]> KEYS *
1) "7515018121727868634"
127.0.0.1:6379[1]> GET 7515018121727868634
"{\"id\":7515018121727868634,\"title\":\"Title 1\",\"text\":\"Text 1\"}"
Download Source Code
$ git clone https://github.com/favtuts/golang-mux-api.git
$ cd golang-mux-api$ git checkout golang-redis-cache
$ go build