SIGABORT when re-using a gfapi.Volume object.

[chrisbecke]

Issue

In an effort to deal with some memory leaks in a volume plugin for docker based on gogfapi, I wanted to reduce the number of gfapi.Volume objects created, but potentially remount them, as docker plugins are long running and I need the plugin to robustly reconnect.

However, re-using a gfapi.Volume object causes an error.

Steps to reproduce.

1. This Dockerfile sets up a golang build environment with gogfapi available:

FROM gluster/glusterfs-client AS builder

RUN yum -y update
RUN yum install -y glusterfs-api glusterfs-api-devel glusterfs-fuse gcc
RUN curl -k https://dl.google.com/go/go1.15.3.linux-amd64.tar.gz | tar xz -C /usr/local

ENV PATH=/usr/local/go/bin:$PATH

2. This docker-compose.yml starts both the builder container, as well as running a gluster server:

version: "3.8"

volumes:
  save1:
  data1:
  lvm1:

services:

  builder:
    image: chrisb/builder
    build:
      context: .
      target: builder
    volumes:
    - ./src:/src
    working_dir: /src

  glusterfs:
    image: gluster/gluster-centos
    hostname: glusterfs
    privileged: true
    environment:
      CGROUP_PIDS_MAX: 0
    volumes:
      - lvm1:/run/lvm
      - save1:/var/lib/glusterd
      - data1:/data

Start glusterfs and make a volume.

docker-compose up -d glusterfs
docker-compose exec glusterfs gluster volume create vol1 glusterfs:/data/vol1
docker-compose exec glusterfs gluster volume start vol1

3. Create the golang program

Create src/main.go on the host system with the following contents:

package main

import (
	"fmt"
	"log"
	"strings"

	"github.com/gluster/gogfapi/gfapi"
)

const gfs_volume = "vol1"
const gfs_hosts = "glusterfs"

func testMount(vol *gfapi.Volume) {
	if err := vol.Mount(); err != nil {
		log.Fatalf("failed %v", err)
	}
	defer vol.Unmount()
}

func main() {
	vol := &gfapi.Volume{}
	hosts := strings.Split(gfs_hosts, ` `)

	if err := vol.Init(gfs_volume, hosts...); err != nil {
		log.Fatalf("failed %v", err)
	}

	for i := 0; i < 2; i++ {
		testMount(vol)
		fmt.Printf(".")
	}
}

4. Execute the program

In a terminal

docker-compose run builder
go run main.go

Observed Behavior

After one successful loop iteration, the program crashes

[root@7063d3435daf reuse-volume]# go run main.go
.free(): invalid pointer
SIGABRT: abort
PC=0x7fe18c41f57f m=0 sigcode=18446744073709551610

goroutine 0 [idle]:
runtime: unknown pc 0x7fe18c41f57f
stack: frame={sp:0x7ffdbd0d2c70, fp:0x0} stack=[0x7ffdbc8d4108,0x7ffdbd0d3140)
00007ffdbd0d2b70:  0000000000000000  0000000000000000 
00007ffdbd0d2b80:  0000000000000000  0000000000000000 
00007ffdbd0d2b90:  0000000000000000  0000000000000000 
00007ffdbd0d2ba0:  0000000000000000  0000000000000000 
00007ffdbd0d2bb0:  0000000000000000  0000000000000000 
00007ffdbd0d2bc0:  0000000000000000  0000000000000000 
00007ffdbd0d2bd0:  0000000000000000  0000000000000000 
00007ffdbd0d2be0:  0000000000000000  0000000000000000 
00007ffdbd0d2bf0:  0000000000000000  0000000000000000 
00007ffdbd0d2c00:  0000000000000000  0000000000000000 
00007ffdbd0d2c10:  0000000000000000  0000000000000000 
00007ffdbd0d2c20:  0000000000000000  0000000000000000 
00007ffdbd0d2c30:  0000000000000000  0000000000000000 
00007ffdbd0d2c40:  0000000000000000  0000000000000000 
00007ffdbd0d2c50:  0000000000000000  0000000000000000 
00007ffdbd0d2c60:  0000000000000000  0000000000000000 
00007ffdbd0d2c70: <0000000000000000  0000000000000000 
00007ffdbd0d2c80:  0000000000000000  0000000000000000 
00007ffdbd0d2c90:  0000000000000000  0000000000000000 
00007ffdbd0d2ca0:  0000000000000000  0000000000000000 
00007ffdbd0d2cb0:  0000000000000000  0000000000000000 
00007ffdbd0d2cc0:  0000000000000000  0000000000000000 
00007ffdbd0d2cd0:  0000000000000000  0000000000000000 
00007ffdbd0d2ce0:  0000000000000000  0000000000000000 
00007ffdbd0d2cf0:  fffffffe7fffffff  ffffffffffffffff 
00007ffdbd0d2d00:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d10:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d20:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d30:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d40:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d50:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d60:  ffffffffffffffff  ffffffffffffffff 
runtime: unknown pc 0x7fe18c41f57f
stack: frame={sp:0x7ffdbd0d2c70, fp:0x0} stack=[0x7ffdbc8d4108,0x7ffdbd0d3140)
00007ffdbd0d2b70:  0000000000000000  0000000000000000 
00007ffdbd0d2b80:  0000000000000000  0000000000000000 
00007ffdbd0d2b90:  0000000000000000  0000000000000000 
00007ffdbd0d2ba0:  0000000000000000  0000000000000000 
00007ffdbd0d2bb0:  0000000000000000  0000000000000000 
00007ffdbd0d2bc0:  0000000000000000  0000000000000000 
00007ffdbd0d2bd0:  0000000000000000  0000000000000000 
00007ffdbd0d2be0:  0000000000000000  0000000000000000 
00007ffdbd0d2bf0:  0000000000000000  0000000000000000 
00007ffdbd0d2c00:  0000000000000000  0000000000000000 
00007ffdbd0d2c10:  0000000000000000  0000000000000000 
00007ffdbd0d2c20:  0000000000000000  0000000000000000 
00007ffdbd0d2c30:  0000000000000000  0000000000000000 
00007ffdbd0d2c40:  0000000000000000  0000000000000000 
00007ffdbd0d2c50:  0000000000000000  0000000000000000 
00007ffdbd0d2c60:  0000000000000000  0000000000000000 
00007ffdbd0d2c70: <0000000000000000  0000000000000000 
00007ffdbd0d2c80:  0000000000000000  0000000000000000 
00007ffdbd0d2c90:  0000000000000000  0000000000000000 
00007ffdbd0d2ca0:  0000000000000000  0000000000000000 
00007ffdbd0d2cb0:  0000000000000000  0000000000000000 
00007ffdbd0d2cc0:  0000000000000000  0000000000000000 
00007ffdbd0d2cd0:  0000000000000000  0000000000000000 
00007ffdbd0d2ce0:  0000000000000000  0000000000000000 
00007ffdbd0d2cf0:  fffffffe7fffffff  ffffffffffffffff 
00007ffdbd0d2d00:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d10:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d20:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d30:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d40:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d50:  ffffffffffffffff  ffffffffffffffff 
00007ffdbd0d2d60:  ffffffffffffffff  ffffffffffffffff 

goroutine 1 [syscall]:
runtime.cgocall(0x4ad8d0, 0xc00003ae18, 0x1)
        /usr/local/go/src/runtime/cgocall.go:133 +0x5b fp=0xc00003ade8 sp=0xc00003adb0 pc=0x406b1b
github.com/gluster/gogfapi/gfapi._C2func_glfs_init(0x182c100, 0x0, 0x0, 0x0)
        _cgo_gotypes.go:383 +0x55 fp=0xc00003ae18 sp=0xc00003ade8 pc=0x4abb35
github.com/gluster/gogfapi/gfapi.(*Volume).Mount.func1(0xc00003af60, 0xc0000b0080, 0x1, 0xc00003ae90)
        /root/go/pkg/mod/github.com/gluster/gogfapi@v0.0.0-20200417143235-cc18ed65b187/gfapi/volume.go:92 +0x55 fp=0xc00003ae50 sp=0xc00003ae18 pc=0x4ac955
github.com/gluster/gogfapi/gfapi.(*Volume).Mount(0xc00003af60, 0x1, 0x8)
        /root/go/pkg/mod/github.com/gluster/gogfapi@v0.0.0-20200417143235-cc18ed65b187/gfapi/volume.go:92 +0x2f fp=0xc00003aea8 sp=0xc00003ae50 pc=0x4ac3af
main.testMount(0xc00003af60)
        /src/reuse-volume/main.go:17 +0x3c fp=0xc00003af08 sp=0xc00003aea8 pc=0x4acb3c
main.main()
        /src/reuse-volume/main.go:32 +0x129 fp=0xc00003af88 sp=0xc00003af08 pc=0x4acd29
runtime.main()
        /usr/local/go/src/runtime/proc.go:204 +0x209 fp=0xc00003afe0 sp=0xc00003af88 pc=0x439009
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc00003afe8 sp=0xc00003afe0 pc=0x467961

rax    0x0
rbx    0x6
rcx    0x7fe18c41f57f
rdx    0x0
rdi    0x2
rsi    0x7ffdbd0d2c70
rbp    0x7ffdbd0d2fc0
rsp    0x7ffdbd0d2c70
r8     0x0
r9     0x7ffdbd0d2c70
r10    0x8
r11    0x246
r12    0x1000
r13    0x7ffdbd0d2ee0
r14    0x10
r15    0x7fe164008000
rip    0x7fe18c41f57f
rflags 0x246
cs     0x33
fs     0x0
gs     0x0
exit status 2