Golang Byte Buffer Empty

2018-02-04
I ran into a problem in golang the other day where my bytes.Buffer wasn't storing the bytes I was writing to it. Turns out it's a simple misunderstanding of golang pointers vs golang values.

I had a struct with a single byte.Buffer in it.

type HtmlRewriter struct {
  buf bytes.Buffer
}

func (w HtmlRewriter) Write(p []byte) (n int, err error) {
  return w.buf.Write(p)
}

It's a simple struct which implements the io.Writer interface.

I was using the value instead of a pointer. If you are using a value instead of a pointer in your structs try using a pointer and see if that fixes the issue for you.

type HtmlRewriter struct {
  buf *bytes.Buffer
}

func (w HtmlRewriter) Write(p []byte) (n int, err error) {
  return w.buf.Write(p)
}

I post this here as I couldn't find the answer by googling for 'Golang Byte Buffer Empty' because it's a very specific example of the broader concept of values vs pointers.

Simply put, values are duplicated when passed to functions as parameters or as receivers. This means any changes you made to those values are not applied to the original value.

Read more about it by searching golang pointers vs values.

Golang HTML Template Partials

2017-09-28

An example of rendering a collection of posts using golang as a web server with it's built in "net/http" and "html/template" packages. In this example we make use of functions in the template and also render one template inside another.

package main

import (
  "html/template"
  "io/ioutil"
  "log"
  "net/http"
  "time"
)

func main() {
  mux := http.NewServeMux()

  server := &http.Server{
    Addr:    ":8080",
    Handler: mux,
  }

  funcs := template.FuncMap{
    "Date": func() string { return time.Now().Format("2006-01-02") },
  }
  layout := initTemplate(template.New("layout"), funcs)
  initTemplate(layout.New("post"), funcs)

  mux.HandleFunc("/", home(layout))

  log.Fatal(server.ListenAndServe())
}

type Post struct {
  Title string
  Body  string
}

func home(layout *template.Template) func(http.ResponseWriter, *http.Request) {
  posts := []Post{
    Post{Title: "Hello World!", Body: "I need to start coming up with something better to say than hello world."},
    Post{Title: "Foo bar", Body: "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent egestas."},
  }
  return func(w http.ResponseWriter, r *http.Request) {
    err := layout.Execute(w, posts)
    if err != nil {
      log.Panic(err)
    }
  }
}

func initTemplate(template *template.Template, funcMap template.FuncMap) *template.Template {
  template.Funcs(funcMap)
  contents, err := ioutil.ReadFile(template.Name() + ".html.tmpl")
  if err != nil {
    log.Panic(err)
  }
  template.Parse(string(contents))
  return template
}
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>jdc</title>
  </head>
  <body>
    {{ range . }}
      {{ template "post" . }}
    {{ end}}
  </body>
</html>
<section>
  <h1>{{ Date }} - {{ .Title }}</h1>
  <p>{{ .Body }}</p>
</section>

There is a bit more going on here compared to the previous example. The two main upgrades are rendering a template within a template, and calling functions in a template.

  funcs := template.FuncMap{
    "Date": func() string { return time.Now().Format("2006-01-02") },
  }

Templates have a template.FuncMap which is a collection of funcs accessible in the template. We must have the template.FuncMap setup before the template is parsed. In the above code snippet we create a simple template.FuncMap with a single function to return a date string.

  layout := initTemplate(template.New("home"), funcs)

We've extracted our template initialisation code into the func initTemplate. The func is called and passed in a new empty template and the template.FuncMap we made earlier. It returns to us a useable *template.Template which we store in layout.

func initTemplate(template *template.Template, funcMap template.FuncMap) *template.Template {
  template.Funcs(funcMap)
  contents, err := ioutil.ReadFile(template.Name() + ".html.tmpl")
  if err != nil {
    log.Panic(err)
  }
  template.Parse(string(contents))
  return template
}

We could use template.ParseFiles and it would save us from having to read the template files explicitly with ioutil.ReadFile but it would name our template as "post.html.tmpl" matching the filename. This makes it a little more clumsy when rendering it by name. I'd prefer {{ template "post" . }} over {{ template "post.html.tmpl" . }}.

The reason we create a new template and pass that into initTemplate instead of creating a template in the func is so that we can reuse it to initialise our layout template and also the post template.

  layout := initTemplate(template.New("home"), funcs)
  initTemplate(layout.New("post"), funcs)

The first template is created with template.New("home") and the second with layout.New("post"). Doing it this way allows us to invoke the "post" template in the "layout" template.

type Post struct {
  Title string
  Body  string
}

Create a simple struct for containing our posts.

func home(layout *template.Template) func(http.ResponseWriter, *http.Request) {

Our func (controller action in rails parlance) home takes a pointer to our layout template, and returns the standard HandlerFunc.

  posts := []Post{
    Post{Title: "Hello World!", Body: "I need to start coming up with something better to say than hello world."},
    Post{Title: "Foo bar", Body: "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent egestas."},
  }

Create a slice of Posts to render.

  return func(w http.ResponseWriter, r *http.Request) {
    err := layout.Execute(w, posts)
    if err != nil {
      log.Panic(err)
    }
  }

Return a HandlerFunc closure which executes our layout template passing in posts. And as always, take care of any errors.

  <body>
    {{ range . }}
      {{ template "post" . }}
    {{ end}}
  </body>

The posts passed into layout.Execute(w, posts) are referenced by '.' in the template, as in {{ range . }}. range loops over the posts, setting dot '.' to each post. We then render the post template passing it each post {{ template "post" . }}.

<section>
  <h1>{{ Date }} - {{ .Title }}</h1>
  <p>{{ .Body }}</p>
</section>

The post template renders each post. {{ Date }} is how our "Date" func in the funcMap is called. {{ .Title }} prints the posts Title field. I'll leave it up to the reader to work out what {{ .Body }} is doing.

Golang HTML Template Hello World

2017-09-26

An example of rendering "Hello World!" using golang as a web server with it's built in "net/http" and "html/template" packages.

package main

import (
  "html/template"
  "log"
  "net/http"
)

func main() {
  mux := http.NewServeMux()

  server := &http.Server{
  Addr:    ":8080",
  Handler: mux,
  }

  layout, err := template.ParseFiles("layout.html.tmpl")
  if err != nil {
    log.Panic(err)
  }

  mux.HandleFunc("/", home(layout))

  log.Fatal(server.ListenAndServe())
}

func home(layout *template.Template) func(http.ResponseWriter, *http.Request) {
  return func(w http.ResponseWriter, r *http.Request) {
  err := layout.Execute(w, nil)
    if err != nil {
      log.Panic(err)
    }
  }
}
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>jdc</title>
  </head>
  <body>
    Hello World!
  </body>
</html>

Hello World!

The result isn't any different than our first example (except perhaps a change in font) but it is a step closer to something more useful to run in production.

package main

import (
  "html/template"
  "log"
  "net/http"
)

We've added one package "html/template" to our packages from the last example. This core library in golang gives us the html templates. We don't use any of the features this gives us in this example, but it sets the stage for something later.


func main() {
  mux := http.NewServeMux()

  server := &http.Server{
  Addr:    ":8080",
  Handler: mux,
  }

As before we create a http.ServeMux and register it to our newly created http.Server.


  layout, err := template.ParseFiles("home.html.tmpl")
  if err != nil {
    log.Panic(err)
  }

Parse the template file we have saved in the same directory as our go file which is named "home.html.tmpl". Store the parsed template as layout and check for errors. You'll often have errors in your templates so make sure to display or panic this error as we have done here.


  mux.HandleFunc("/", home(layout))

Pass our newly created template to the home function and register it to the "/" path.


  log.Fatal(server.ListenAndServe())
}

Open the port and start listening for requests.


func home(layout *template.Template) func(http.ResponseWriter, *http.Request) {

There is a bit going on here if you're new to golang. We've defined a new function home which accepts one argument of type *template.Template (pointer to template.Template). The function home also returns a single argument of type func(http.ResponseWriter, *http.Request), that is, it returns a function which takes two arguments. This is a common function signature in the "net/http" package.


  return func(w http.ResponseWriter, r *http.Request) {

Our new function home requires us to return a func, so here we define and return one.


  err := layout.Execute(w, nil)
    if err != nil {
      log.Panic(err)
    }
  }
}

We call Execute(w, nil) on our passed in layout which will write the template to the http.ResponseWriter. Ignore the nil for now, we'll use that later to pass in variables. And as always, grab the error and do something useful with it.

We should call the http.ResponseWriter's WriteHeader function with 200 but golang does this for us if we don't.

Golang Web Server Hello World

2017-09-25

An example of rendering "Hello World!" using golang as a web server with it's built in "net/http" package.

package main

import (
  "log"
  "net/http"
)

func main() {
  mux := http.NewServeMux()

  server := &http.Server{
    Addr:    ":8080",
    Handler: mux,
  }

  mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello World!"))
  })

  log.Fatal(server.ListenAndServe())
}

Hello World!

That is perhaps a little more than strictly necessary for the most basic web server hello world, but I think it's a more useful example which you can build on.

package main

The first golang file is always package main. It has a function main() which will be run when you type go run hello_world.go.

import (
  "log"
  "net/http"
)

Import the packages "log" and "net/http" from golang's standard library.

func main() {

Define the main function which will be executed when this program is run.

  mux := http.NewServeMux()

Mux is an abbreviation for multiplexing. In our context it is what we register our url paths on, giving it functions to call for each. We create a new http.ServeMux and assign it to the variable mux.

  server := &http.Server{
    Addr:    ":8080",
    Handler: mux,
  }

Create a server listening on port 8080 and using the mux we created earlier to handle web requests.

  mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello World!"))
  })

We register the path "/" on our mux and give it a function to respond to requests with. Now when our server gets a request for the path "/" it will call this function to write the response.

  log.Fatal(server.ListenAndServe())

Start the server and log any errors if the server returns one.

Wordpress To Golang

2017-09-24

Less tech is better, so when reaching for a blog I skipped wordpress and picked up golang. A couple years ago Ruby on Rails was a consideration but using that would push me to use a larger more expensive virtual machine compared to using golang.

Wordpress runs the internet, at least a significant share of it. The wiki page mentions 27.5% of the top 10 million websites as of February 2017 which is impressive. My biggest gripe with Wordpress is it runs on php, which puts me off significantly. I've not used it much but it doesn't have a good reputation, at least, not in the Ruby community. I want my blog to be a reflection of me and my interest in software, so the tech which runs it should echo that.

Ruby is a great technology, it makes for happier devs just like its creator Yukihiro Matsumoto wanted.

I hope to see Ruby help every programmer in the world to be productive, and to enjoy programming, and to be happy. That is the primary purpose of Ruby language.

Yukihiro Matsumoto 2008 Google Tech Talk 'Ruby 1.9'

Ruby can scale, I'm not going to say otherwise. However it is at the expense of increasing infrastructure complexity. If your app is big and complex then it is worth using ruby and making the effort to scale it. If your app is simple then the complexity cost of the infrastructure will outweigh the benefit of using Ruby. Your developers will spend most of the time conforming to the infrastructure, the happiness they get from using Ruby will be shadowed.

Another strike against Ruby is that I know Ruby. I won't be learning anything by using it to run my blog. If you are new to web development and want to create your own personal brand then perhaps using Ruby to build your own blog would make sense. It depends what you want out of it.

I could have chosen Elixir, or Rust or Node.js. I chose golang. I read about the creation of golang and it's authors pain points for existing languages. What really stood out to me was the focus on simplicity. I've only been in software for nearing 10 years at this point but one thing I notice a lot and I was guilty of early on is ego. That is, writing complicated code to show off and prove oneself. It's not necessarily from a bad place, imposter syndrome is rife in the tech industry, I certainly feel it most of the time. Whatever the cause, complicated code for complicated sake doesn't help anyone. Golang was created to minimise the ability to write clever code, opting for clear and simple syntax with few options to do something in more than one way. This sometimes means you end up writing more code, but I strongly believe it is worth it.

Here is my blog. It's running golang 1.9. It's paired with Postgresql 9.6. It's run on $5 vultr.com VM's located in 4 different data centres around the world. To find the reader's closest server I use ns1.com's free DNS hosting. SSL certs are issued for free by letsencrypt.org.