There are different ways of debugging a Go program. You could use print statements to view the values stored in your variables during the execution or you could use a debugger. Some IDEs, like GoLand, offer this functionality by default but it is also interesting to see how to use the integrated solutions from the command line. Delve offers both command line clients and an API that can be used to integrate it to other tools (which is actually what GoLand uses under the hood).
Getting started
The easiest way to install Delve is to use the standard go get command:
$ go get github.com/go-delve/delve/cmd/dlv
Once installed you can use it to debug your Go application. I am going to use a simple Go program that you can find in my Github repository (under delve-debug). It has a main function with two nested loops that increment the value of a variable and two auxiliary functions that don’t do much other than call each other.
🖥️ The full code is on my Github repo: https://github.com/efrag/blog-posts
Navigate to the directory that has your main.go file and type
$ cd github.com/efrag/blog-posts/delve-debug/ $ dlv debug (dlv)
This will bring up the (dlv) terminal. Keep in mind that this will compile and build your program which is now associated with a running process. Typing help will bring up all available options and help <command> will display information about the specific command.
Breakpoints
One of the most used commands is the continue one. If we go ahead and run it we will see the following:
(dlv) continue Process 237146 has exited with status 0
which just means that are program run successfully and there is nothing else to see here 👀. Happy as we are that are program finished successfully we probably wanted to see more information about what happens. The way to do that is by setting breakpoints, like we would in our IDE but a bit more manual 🙂.
Add breakpoints
So let’s set some breakpoints. We will set one breakpoint on line 25 (the outer most loop that we have in our main function) and one on line 27 (the line where we change the value of k).
(dlv) restart Process restarted with PID 238493
The restart here is going to restart the execution of our program (required as we have completed the current run and there is nothing else delve can do at this point).
(dlv) break main.go:25 Breakpoint 1 set at 0x467aaa for main.main() ./main.go:25 (dlv) break main.go:27 Breakpoint 2 set at 0x467ad5 for main.main() ./main.go:27
The following two commands are the break commands that create breakpoints on lines 25 and 27 of the main.go file that we have. That’s it 🎉!
Listing existing breakpoints
The breakpoints command allows us to see what is already set and we can see our 2 breakpoints set on lines 25 and 27.
(dlv) breakpoints Breakpoint 1 at 0x467aaa for main.main() ./main.go:25 (0) Breakpoint 2 at 0x467ad5 for main.main() ./main.go:27 (0)
Removing breakpoints
The clear command allows us to delete an endpoint that we have already set. It takes one argument which is the numeric value of the endpoint that we saw previously. Running the following and then listing the breakpoints again will only show us the one endpoint that we have remaining on line 27.
(dlv) clear 1 Breakpoint 1 cleared at 0x467aaa for main.main() ./main.go:25 (dlv) breakpoints Breakpoint 2 at 0x467ad5 for main.main() ./main.go:27 (0)
Debugging
Let’s start our debugging and see what else we can read from delve. We saw previously that using the continue command will execute the code to the next available breakpoint (and since we had none it run the entire program). Now that we do have a breakpoint though we would expect that the execution of our program will stop at the point where we set our breakpoint. Let’s try it.
(dlv) continue
> main.main() ./main.go:27 (hits goroutine(1):1 total:1) (PC: 0x467ad5)
22: func main() {
23: k := 100
24:
25: for i := 1; i <= 10; i++ {
26: for j := i; j <= 10; j++ {
=> 27: k += i
28: }
29: }
30:
31: _ = nest1(1)
32: }
(dlv)The output of continue now is quite different. It has run the program up until the line 27, it has printed the lines around our breakpoint and has returned to the (dlv) prompt in our terminal. If we wish the execution to resume again we should give the continue command again.
So what else would we like to do now? It would super interesting to see the values of our variables at this point of the execution. We can do that by using another command called print.
(dlv) print i 1 (dlv) print j 1 (dlv) print k 100
If we wanted to move to the next point we would do continue again and get a similar output with above. And if we wanted to see the values of i, j and k we would need to type in again the print commands that is a bit tedious to repeat. Luckily delve provides a nice command to help us out with this.
(dlv) on 2 print i (dlv) on 2 print j (dlv) on 2 print k
These commands instruct delve to run the print commands every time we hit breakpoint 2. So running this again we now get the following output with the values of our variables already printed out 🎉!
(dlv) continue
> main.main() ./main.go:27 (hits goroutine(1):2 total:2) (PC: 0x467ad5)
i: 1
j: 2
k: 101
22: func main() {
23: k := 100
24:
25: for i := 1; i <= 10; i++ {
26: for j := i; j <= 10; j++ {
=> 27: k += i
28: }
29: }
30:
31: _ = nest1(1)
32: }
(dlv)Stack traces
Another very useful thing that we can see in our terminal is the stack trace for the code that we are currently executing.
(dlv) stack 0 0x0000000000467ad5 in main.main at ./main.go:27 1 0x0000000000435b2f in runtime.main at /usr/lib/go-1.15/src/runtime/proc.go:204 2 0x0000000000464261 in runtime.goexit at /usr/lib/go-1.15/src/runtime/asm_amd64.s:1374 (dlv)
Let’s see something going a bit deeper than just hitting our main function. I will set up a breakpoint inside one of the auxiliary functions mentioned above and run the program up until that point.
(dlv) break main.go:19
Breakpoint 3 set at 0x4679dc for main.nest2() ./main.go:19
(dlv) clear 2 # removing to quickly go to the next breakpoint just created
Breakpoint 2 cleared at 0x467ad5 for main.main() ./main.go:27
(dlv) continue
> main.nest2() ./main.go:19 (hits goroutine(1):1 total:1) (PC: 0x4679dc)
14: ret = append(ret, 2)
15:
16: i1 += 1
17: i2 += 1
18:
=> 19: return ret
20: }
21:
22: func main() {
23: k := 100
24:The output of continue is different now, since we moved to a different part of the program. Asking to see the stack trace again will show us more information about the depth of the calls we have made
(dlv) stack 0 0x00000000004679dc in main.nest2 at ./main.go:19 1 0x00000000004678b2 in main.nest1 at ./main.go:6 2 0x0000000000467b13 in main.main at ./main.go:31 3 0x0000000000435b2f in runtime.main at /usr/lib/go-1.15/src/runtime/proc.go:204 4 0x0000000000464261 in runtime.goexit at /usr/lib/go-1.15/src/runtime/asm_amd64.s:1374
Now the trace tells us, that apart from the core go calls and the call to main we have 2 more calls in our stack trace, the first one is from the main function to the nest1 function and from there to the nest2 function.
And that’s it – you can now use delve from the command line and explore all available options. You can get information about the threads and goroutines of your program, on the fly change the values of your variables and even inspect the contents of your CPU registers!
[Bonus] CPU registers
This is a bit more of a deep dive on how the processes store information about the current execution of the program, so feel free to skip but I thought it was so interesting to see that I had to put a reference in 😎. By typing regs in the terminal you can get a dump of the CPU registers at this point in time of the execution of your program. The output will look something like this.
(dlv) regs
Rip = 0x0000000000467ad5
Rsp = 0x000000c000030750
Rax = 0x0000000000000002
Rbx = 0x0000000000000000
Rcx = 0x000000c000000180
Rdx = 0x0000000000483370
Rsi = 0x0000000000000060
Rdi = 0x0000000000000000
Rbp = 0x000000c000030778
R8 = 0xffffffffffffffff
R9 = 0x7fffffffffffffff
R10 = 0x0000000004000000
R11 = 0x00000000004dd460
R12 = 0x0000000000203000
R13 = 0x0000000000000000
R14 = 0x0000000000000058
R15 = 0x0000000000000004
Rflags = 0x0000000000000293 [CF AF SF IF IOPL=0]
Es = 0x0000000000000000
Cs = 0x0000000000000033
Ss = 0x000000000000002b
Ds = 0x0000000000000000
Fs = 0x0000000000000000
Gs = 0x0000000000000000
Fs_base = 0x00000000004c71f0
Gs_base = 0x0000000000000000Where the Rip register is the Instruction Pointer that holds the address of the current instruction being executed. If you remember from before we had this line being printed once we hit continue on our breakpoint.
(dlv) continue > main.main() ./main.go:27 (hits goroutine(1):2 total:2) (PC: 0x467ad5)
If you notice at the end there is a (PC: 0x467ad5) which is actually the value of the Rip register!