Ways of Debugging HTTP Transactions in Golang
As a software engineer, most of the time you spend on debugging. You probably interact with external services via HTTP API to deal with data transactions you might need on your application, so it’s gonna be very crucial to understand exactly what happens on those transactions. So, here are some ways you could try to see what’s going on in those HTTP transactions in Golang.
GODEBUG Environment variable
Without modifying your code to add checkpoints here and there, you can utilize GODEBUG environment variable to let golang spill out the HTTP transaction your app is making. Some of the flags I usually use to do this are:
- GODEBUG=http1debug=1
- GODEBUG=http1debug=2
- GODEBUG=http2debug=1
- GODEBUG=http2debug=2
http1debug and http2debug are the flags you need to see the logs about the HTTP transaction to its respective HTTP version. 1 and 2 represents the verbosity you want to see on the logs. 2 would be much more detailed compared to 1 . However, the caveat is that it might give you too much information that you don’t need. It doesn’t allow you to cherry-pick information you want to see on the transaction.
Logging before and after making an HTTP transaction
Obviously, you have a lot of control over this because you just need to add more verbosity on your own code. You can just simply fmt.Println on the request you are about to make and then fmt.Println again after the said request is done. While you can cherry-pick the verbosity of the information you want to see, it could potentially make your code a bit dirty and accidentally pushed. It could be leveraged with something like logrus combined with how you handle APP_LOG_LEVEL you set. So if the level is info, then your debugging piece won’t noise the whole app logging.
Extend http.RoundTrip implementation
This step is very similar to the above just with another extra step. So, instead of logging on to your use case or repository code that will do external service calls, you can do the logging on to the wrapper instead. This way is more isolated as you only do an extension of http.RoundTrip . For example, say you have this kind of call in your code
func GetCats(filter *CatFilter) ([]*CatResponse, error) {
if filter == nil {
filter = &CatFilter{}
}
// even if filter is null. keep process it.
req, err := http.NewRequestWithContext(ctx, "GET", host+"/api/cats", nil)
if err != nil {
return nil, err
}
limit := defaultLimit
u := req.URL
q := u.Query()
if filter.Tags != "" {
q.Set("tags", filter.Tags)
}
if filter.Skip > 0 {
q.Set("skip", strconv.Itoa(filter.Skip))
}
if filter.Limit > 0 {
limit = filter.Limit
}
q.Set("limit", strconv.Itoa(limit))
u.RawQuery = q.Encode()
fmt.Println("request", req.URL.String()) // debugging piece
res, err := c.Client.Do(req)
if err != nil {
return nil, err
}
if res.StatusCode >= 500 && res.StatusCode <= 504 {
return nil, errors.New("server error")
}
defer res.Body.Close()
var bufReader bytes.Buffer
tee := io.TeeReader(res.Body, &bufReader)
var (
result []*CatResponse
)
if err = json.NewDecoder(tee).Decode(&result); err != nil {
return nil, err
}
fmt.Println("response", bufReader.String()) //debugging piece
return result, nil
}You could modify it a bit by going like this
type roundtripLogger struct {
}
func (r roundtripLogger) RoundTrip(request *http.Request) (*http.Response, error) {
fmt.Println("request", request.URL.String())
res, err := http.DefaultTransport.RoundTrip(request)
var bufReader bytes.Buffer
io.Copy(&bufReader, res.Body)
fmt.Println("response", bufReader.String())
// res.Body is already closed. you need to make a copy again to pass it for the code
res.Body = io.NopCloser(bytes.NewReader(bufReader.Bytes()))
return res, err
}
func GetCats(filter *CatFilter) ([]*CatResponse, error) {
if filter == nil {
filter = &CatFilter{}
}
// even if filter is null. keep process it.
req, err := http.NewRequestWithContext(ctx, "GET", host+"/api/cats", nil)
if err != nil {
return nil, err
}
limit := defaultLimit
u := req.URL
q := u.Query()
if filter.Tags != "" {
q.Set("tags", filter.Tags)
}
if filter.Skip > 0 {
q.Set("skip", strconv.Itoa(filter.Skip))
}
if filter.Limit > 0 {
limit = filter.Limit
}
q.Set("limit", strconv.Itoa(limit))
u.RawQuery = q.Encode()
cli := &http.Client{Transport: roundtripLogger{}}
res, err := cli.Do(req)
if err != nil {
return nil, err
}
if res.StatusCode >= 500 && res.StatusCode <= 504 {
return nil, errors.New("server error")
}
defer res.Body.Close()
var (
result []*CatResponse
)
if err = json.NewDecoder(res.Body).Decode(&result); err != nil {
return nil, err
}
return result, nil
}This way, your code is a bit cleaner. I also make a simple go HTTP client utility that can help you add an interceptor and/or transformer on each call.
func GetCats(filter *CatFilter) ([]*CatResponse, error) {
// no client declaration, use the object from the struct
if filter == nil {
filter = &CatFilter{}
}
// even if filter is null. keep process it.
req, err := http.NewRequestWithContext(ctx, "GET", host+"/api/cats", nil)
if err != nil {
return nil, err
}
limit := defaultLimit
u := req.URL
q := u.Query()
if filter.Tags != "" {
q.Set("tags", filter.Tags)
}
if filter.Skip > 0 {
q.Set("skip", strconv.Itoa(filter.Skip))
}
if filter.Limit > 0 {
limit = filter.Limit
}
q.Set("limit", strconv.Itoa(limit))
u.RawQuery = q.Encode()
client := httpclient.Standard()
client.AddInterceptor(interceptor.RequestLogger(logrus.New())).
AddTransformer(transformer.ResponseLogger(logrus.New()))
res, err := client.Do(req)
if err != nil {
return nil, err
}
defer res.Body.Close()
var (
result []*CatResponse
)
if err = json.NewDecoder(res.Body).Decode(&result); err != nil {
return nil, err
}
return result, nil
}Alright, that’s all I know about HTTP transaction debugging. If you have other ideas on how to do that, it would be great if you leave them in the response so we can learn more about Golang together. That’s all for the story of this week, See you on the next story!
References:
