Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSDK-4106: Restructure lidar name and data rate to use map #213

Closed
wants to merge 43 commits into from
Closed
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
3fe89df
go config lidar rename
pstrutz Jul 21, 2023
00badc5
more go config changes
pstrutz Jul 21, 2023
c6601b4
go config passing tests with new sensors naming
pstrutz Jul 21, 2023
4c276e3
fixed naming in integration tests
pstrutz Jul 21, 2023
ba1a345
structural changes to lidar
pstrutz Jul 21, 2023
41f06ad
initial changes
pstrutz Jul 21, 2023
e4af834
more name changes
pstrutz Jul 24, 2023
a68ac81
hotfix-add-nlopt (#212)
nicksanford Jul 21, 2023
14f5c56
RSDK-3718 - cartographer appends to infinitely growing vector fix (#209)
kim-mishra Jul 24, 2023
18c9d7b
working on resolving issues
pstrutz Jul 24, 2023
6d688bc
working sensor tests
pstrutz Jul 24, 2023
bee9d7f
passing sensorprocess tests
pstrutz Jul 24, 2023
7c7abfe
[RSDK-4148] remove-makefile-and-gh-action-full-mod (#216)
nicksanford Jul 24, 2023
8608484
Revert "[RSDK-4148] remove-makefile-and-gh-action-full-mod" (#220)
nicksanford Jul 24, 2023
d7d8c5f
vc changes
pstrutz Jul 24, 2023
fc03df7
restore unnecessary change
pstrutz Jul 25, 2023
dc7adb3
lint fixes
pstrutz Jul 25, 2023
09c0bda
go config lidar rename
pstrutz Jul 21, 2023
339ae73
more go config changes
pstrutz Jul 21, 2023
8679e01
go config passing tests with new sensors naming
pstrutz Jul 21, 2023
ac1eccf
fixed naming in integration tests
pstrutz Jul 21, 2023
260e94a
structural changes to lidar
pstrutz Jul 21, 2023
c456d37
initial changes
pstrutz Jul 21, 2023
2304abc
more name changes
pstrutz Jul 24, 2023
128c20b
working on resolving issues
pstrutz Jul 24, 2023
281d4a0
working sensor tests
pstrutz Jul 24, 2023
7e3386a
passing sensorprocess tests
pstrutz Jul 24, 2023
a4e6284
vc changes
pstrutz Jul 24, 2023
01751f8
restore unnecessary change
pstrutz Jul 25, 2023
0241490
lint fixes
pstrutz Jul 25, 2023
c1edb8a
Merge remote-tracking branch 'refs/remotes/origin/RSDK-4106' into RSD…
pstrutz Jul 25, 2023
def1699
resolve issues
pstrutz Jul 25, 2023
1c2fc0d
small fix
pstrutz Jul 25, 2023
f1f8b9f
respond to comments
pstrutz Jul 25, 2023
bbf95a7
fixes
pstrutz Jul 25, 2023
2fa3e28
config fixes
pstrutz Jul 26, 2023
9b389be
Merge branch 'main' of github.com:viamrobotics/viam-cartographer into…
pstrutz Jul 26, 2023
de90d37
clean up config test
pstrutz Jul 26, 2023
36cc0a3
should work now
pstrutz Jul 26, 2023
52bb8b7
capi
pstrutz Jul 26, 2023
647e244
capi fix
pstrutz Jul 26, 2023
699cd47
resolve testing issues
pstrutz Jul 26, 2023
21da3bc
debugging
pstrutz Jul 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions cartofacade/capi.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ const (

// CartoConfig contains config values from app
type CartoConfig struct {
Sensors []string
Camera map[string]string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Camera map[string]string
CameraName string

MapRateSecond int
DataDir string
ComponentReference string
Expand Down Expand Up @@ -179,7 +179,7 @@ func NewCarto(cfg CartoConfig, acfg CartoAlgoConfig, vcl CartoLibInterface) (Car
return Carto{}, err
}

status = C.free_bstring_array(vcc.sensors, C.size_t(len(cfg.Sensors)))
status = C.free_bstring_array(vcc.sensors, C.size_t(1))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets change the C API to take a lidar & optionally an imu, rather than having a sensor list

if status != C.BSTR_OK {
return Carto{}, errors.New("unable to free memory for sensor list")
}
Expand Down Expand Up @@ -354,15 +354,15 @@ func getConfig(cfg CartoConfig) (C.viam_carto_config, error) {
vcc := C.viam_carto_config{}

// create pointer to bstring which can represent a list of sensors
sz := len(cfg.Sensors)
sz := 1
pSensor := C.alloc_bstring_array(C.size_t(sz))
if pSensor == nil {
return C.viam_carto_config{}, errors.New("unable to allocate memory for sensor list")
}
sensorSlice := unsafe.Slice(pSensor, sz)
for i, sensor := range cfg.Sensors {
sensorSlice[i] = goStringToBstring(sensor)
}

sensorSlice[0] = goStringToBstring(cfg.Camera["name"])

lidarCfg, err := toLidarConfig(cfg.LidarConfig)
if err != nil {
return C.viam_carto_config{}, err
Expand Down
4 changes: 2 additions & 2 deletions cartofacade/testhelpers.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ func GetTestConfig(sensor string) (CartoConfig, string, error) {
}

return CartoConfig{
Sensors: []string{sensor, "imu"},
Camera: map[string]string{"name": "test_lidar", "data_freq_hz": "20"},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cartographer doesn't need to know the data_frequency_hz does it?

MapRateSecond: 5,
DataDir: dir,
ComponentReference: "component",
Expand All @@ -23,7 +23,7 @@ func GetTestConfig(sensor string) (CartoConfig, string, error) {
// GetBadTestConfig gets a sample config for testing purposes that will cause a failure.
func GetBadTestConfig() CartoConfig {
return CartoConfig{
Sensors: []string{"rplidar", "imu"},
Camera: map[string]string{"name": "rplidar", "data_freq_hz": "20"},
LidarConfig: TwoD,
}
}
Expand Down
41 changes: 21 additions & 20 deletions config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
package config

import (
"strconv"

"github.com/edaniels/golog"
"github.com/pkg/errors"
"go.viam.com/utils"
Expand All @@ -14,19 +16,29 @@ func newError(configError string) error {

// Config describes how to configure the SLAM service.
type Config struct {
Sensors []string `json:"sensors"`
Camera map[string]string `json:"camera"`
ConfigParams map[string]string `json:"config_params"`
DataDirectory string `json:"data_dir"`
DataRateMsec int `json:"data_rate_msec"`
MapRateSec *int `json:"map_rate_sec"`
}

var errSensorsMustNotBeEmpty = errors.New("\"sensors\" must not be empty")
var errCameraMustHaveName = errors.New("\"camera[name]\" is required")

// Validate creates the list of implicit dependencies.
func (config *Config) Validate(path string) ([]string, error) {
if config.Sensors == nil || len(config.Sensors) < 1 {
return nil, utils.NewConfigValidationError(path, errSensorsMustNotBeEmpty)
_, ok := config.Camera["name"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_, ok := config.Camera["name"]
cameraName, ok := config.Camera["name"]

if !ok {
return nil, utils.NewConfigValidationError(path, errCameraMustHaveName)
}
dataFreqHz, ok := config.Camera["data_freq_hz"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scope doc says this is data_frequency_hz

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dataFreqHz, ok := config.Camera["data_freq_hz"]
cameraDataFreqHzStr, ok := config.Camera["data_frequency_hz"]

if ok {
dataFreqHz, err := strconv.Atoi(dataFreqHz)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dataFreqHz, err := strconv.Atoi(dataFreqHz)
dataFreqHz, err := strconv.Atoi(cameraDataFreqHzStr)

if err != nil {
return nil, errors.New("data_freq_hz must only contain digits")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should specify we are talking about camera[data_frequency_hz]

}
if dataFreqHz < 0 {
return nil, errors.New("cannot specify data_freq_hz less than zero")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should specify we are talking about camera[data_frequency_hz]

}
}

if config.ConfigParams["mode"] == "" {
Expand All @@ -37,30 +49,19 @@ func (config *Config) Validate(path string) ([]string, error) {
return nil, utils.NewConfigValidationFieldRequiredError(path, "data_dir")
}

if config.DataRateMsec < 0 {
return nil, errors.New("cannot specify data_rate_msec less than zero")
}

if config.MapRateSec != nil && *config.MapRateSec < 0 {
return nil, errors.New("cannot specify map_rate_sec less than zero")
}

deps := config.Sensors
deps := []string{config.Camera["name"]}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
deps := []string{config.Camera["name"]}
deps := []string{cameraName}


return deps, nil
}

// GetOptionalParameters sets any unset optional config parameters to the values passed to this function,
// and returns them.
func GetOptionalParameters(config *Config,
defaultDataRateMsec, defaultMapRateSec int, logger golog.Logger,
) (int, int) {
dataRateMsec := config.DataRateMsec
if config.DataRateMsec == 0 {
dataRateMsec = defaultDataRateMsec
logger.Debugf("no data_rate_msec given, setting to default value of %d", defaultDataRateMsec)
}

func GetOptionalParameters(config *Config, defaultMapRateSec int, logger golog.Logger,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where are we deriving the default lidar data rate?

) int {
mapRateSec := 0
if config.MapRateSec == nil {
logger.Debugf("no map_rate_sec given, setting to default value of %d", defaultMapRateSec)
Expand All @@ -69,5 +70,5 @@ func GetOptionalParameters(config *Config,
mapRateSec = *config.MapRateSec
}

return dataRateMsec, mapRateSec
return mapRateSec
}
69 changes: 48 additions & 21 deletions config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ func TestValidate(t *testing.T) {
model := resource.DefaultModelFamily.WithModel("test")
cfgService := resource.Config{Name: "test", API: slam.API, Model: model}
_, err := newConfig(cfgService)
test.That(t, err, test.ShouldBeError, newError("error validating \"services.slam.attributes.fake\": \"sensors\" must not be empty"))
test.That(t, err, test.ShouldBeError, newError("error validating \"services.slam.attributes.fake\": \"camera[name]\" is required"))
})

t.Run("Simplest valid config", func(t *testing.T) {
Expand All @@ -28,13 +28,13 @@ func TestValidate(t *testing.T) {
})

t.Run("Config without required fields", func(t *testing.T) {
requiredFields := []string{"data_dir", "sensors"}
requiredFields := []string{"data_dir", "camera"}
dataDirErr := utils.NewConfigValidationFieldRequiredError(testCfgPath, requiredFields[0])
sensorsErr := utils.NewConfigValidationError(testCfgPath, errSensorsMustNotBeEmpty)
cameraErr := utils.NewConfigValidationError(testCfgPath, errCameraMustHaveName)

expectedErrors := []error{
newError(dataDirErr.Error()),
newError(sensorsErr.Error()),
newError(cameraErr.Error()),
}
for i, requiredField := range requiredFields {
logger.Debugf("Testing SLAM config without %s\n", requiredField)
Expand All @@ -52,6 +52,15 @@ func TestValidate(t *testing.T) {
test.That(t, err, test.ShouldBeError, newError(utils.NewConfigValidationFieldRequiredError(testCfgPath, "config_params[mode]").Error()))
})

t.Run("Config without camera name", func(t *testing.T) {
// Test for missing camera name attribute
logger.Debug("Testing SLAM config without camera[name]")
cfgService := makeCfgService()
delete(cfgService.Attributes["camera"].(map[string]string), "name")
_, err := newConfig(cfgService)
test.That(t, err, test.ShouldBeError, newError(utils.NewConfigValidationFieldRequiredError(testCfgPath, "camera[name]").Error()))
})

t.Run("Config with invalid parameter type", func(t *testing.T) {
cfgService := makeCfgService()
cfgService.Attributes["data_dir"] = true
Expand All @@ -63,21 +72,39 @@ func TestValidate(t *testing.T) {
test.That(t, err, test.ShouldBeNil)
})

t.Run("Config with invalid camera data_freq_hz", func(t *testing.T) {
cfgService := makeCfgService()
cfgService.Attributes["camera"] = map[string]string{
"name": "a",
"data_freq_hz": "twenty",
}
_, err := newConfig(cfgService)
test.That(t, err, test.ShouldBeError, newError("data_freq_hz must only contain digits"))
})

t.Run("Config with out of range values", func(t *testing.T) {
cfgService := makeCfgService()
cfgService.Attributes["data_rate_msec"] = -1
cfgService.Attributes["camera"] = map[string]string{
"name": "a",
"data_freq_hz": "-1",
}
_, err := newConfig(cfgService)
test.That(t, err, test.ShouldBeError, newError("cannot specify data_rate_msec less than zero"))
cfgService.Attributes["data_rate_msec"] = 1
test.That(t, err, test.ShouldBeError, newError("cannot specify data_freq_hz less than zero"))
cfgService.Attributes["camera"] = map[string]string{
"name": "a",
"data_freq_hz": "1",
}
cfgService.Attributes["map_rate_sec"] = -1
_, err = newConfig(cfgService)
test.That(t, err, test.ShouldBeError, newError("cannot specify map_rate_sec less than zero"))
})

t.Run("All parameters e2e", func(t *testing.T) {
cfgService := makeCfgService()
cfgService.Attributes["sensors"] = []string{"a", "b"}
cfgService.Attributes["data_rate_msec"] = 1001
cfgService.Attributes["camera"] = map[string]string{
"name": "a",
"data_freq_hz": "20",
}
cfgService.Attributes["map_rate_sec"] = 1002

cfgService.Attributes["config_params"] = map[string]string{
Expand All @@ -88,8 +115,7 @@ func TestValidate(t *testing.T) {
cfg, err := newConfig(cfgService)
test.That(t, err, test.ShouldBeNil)
test.That(t, cfg.DataDirectory, test.ShouldEqual, cfgService.Attributes["data_dir"])
test.That(t, cfg.Sensors, test.ShouldResemble, cfgService.Attributes["sensors"])
test.That(t, cfg.DataRateMsec, test.ShouldEqual, cfgService.Attributes["data_rate_msec"])
test.That(t, cfg.Camera, test.ShouldResemble, cfgService.Attributes["camera"])
test.That(t, *cfg.MapRateSec, test.ShouldEqual, cfgService.Attributes["map_rate_sec"])
test.That(t, cfg.ConfigParams, test.ShouldResemble, cfgService.Attributes["config_params"])
})
Expand All @@ -104,7 +130,10 @@ func makeCfgService() resource.Config {
"mode": "test mode",
}
cfgService.Attributes["data_dir"] = "path"
cfgService.Attributes["sensors"] = []string{"a"}
cfgService.Attributes["camera"] = map[string]string{
"name": "a",
"data_freq_hz": "20",
}
return cfgService
}

Expand All @@ -113,32 +142,30 @@ func TestGetOptionalParameters(t *testing.T) {

t.Run("Pass default parameters", func(t *testing.T) {
cfgService := makeCfgService()
cfgService.Attributes["sensors"] = []string{"a"}
cfgService.Attributes["camera"] = map[string]string{"name": "a"}
cfg, err := newConfig(cfgService)
test.That(t, err, test.ShouldBeNil)
dataRateMsec, mapRateSec := GetOptionalParameters(
mapRateSec := GetOptionalParameters(
cfg,
1001,
1002,
logger)
test.That(t, dataRateMsec, test.ShouldEqual, 1001)
test.That(t, mapRateSec, test.ShouldEqual, 1002)
})

t.Run("Return overrides", func(t *testing.T) {
cfgService := makeCfgService()
cfgService.Attributes["sensors"] = []string{"a"}
cfgService.Attributes["camera"] = map[string]string{
"name": "a",
"data_freq_hz": "1",
}
cfg, err := newConfig(cfgService)
two := 2
cfg.DataRateMsec = 1
cfg.MapRateSec = &two
test.That(t, err, test.ShouldBeNil)
dataRateMsec, mapRateSec := GetOptionalParameters(
mapRateSec := GetOptionalParameters(
cfg,
1001,
1002,
logger)
test.That(t, dataRateMsec, test.ShouldEqual, 1)
test.That(t, mapRateSec, test.ShouldEqual, 2)
})
}
Expand Down
8 changes: 4 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ require (
github.com/rhysd/actionlint v1.6.24
github.com/viamrobotics/gostream v0.0.0-20230609200515-c5d67c29ed25
go.opencensus.io v0.24.0
go.uber.org/multierr v1.11.0
go.uber.org/zap v1.24.0
go.viam.com/api v0.1.156
go.viam.com/rdk v0.5.1-0.20230719205427-c10eab2aa624
go.viam.com/test v1.1.1-0.20220913152726-5da9916c08a2
go.viam.com/utils v0.1.38
google.golang.org/grpc v1.56.0
google.golang.org/protobuf v1.30.0
)

require (
Expand Down Expand Up @@ -278,8 +278,6 @@ require (
go.mongodb.org/mongo-driver v1.12.0-prerelease.0.20221109213319-d3466eeae7a7 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/goleak v1.2.1 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.24.0 // indirect
goji.io v2.0.2+incompatible // indirect
golang.org/x/crypto v0.10.0 // indirect
golang.org/x/exp v0.0.0-20230522175609-2e198f4a06a1 // indirect
Expand All @@ -300,6 +298,8 @@ require (
google.golang.org/genproto v0.0.0-20230530153820-e85fd2cbaebc // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230530153820-e85fd2cbaebc // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230530153820-e85fd2cbaebc // indirect
google.golang.org/grpc v1.56.0 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
gopkg.in/src-d/go-billy.v4 v4.3.2 // indirect
Expand Down
Loading
Loading