Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expose SRIOV information #230

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1115,6 +1115,67 @@ information
`ghw.TopologyNode` struct if you'd like to dig deeper into the NUMA/topology
subsystem

### SRIOV

*This API is PROVISIONAL! ghw will try hard to not make breaking changes to this API, but still users are advice this new API is not
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend removing this. :) The interfaces we publish are specific to a Git tag or SHA1 per Go package versioning and semver behaviour.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problem! will remove.

declared stable yet. We expect to declare it stable with ghw version 1.0.0*

SRIOV (Single-Root Input/Output Virtualization) is a class of PCI devices that ghw models explicitly, like gpus.

```go
package main

import (
"fmt"

"github.com/jaypipes/ghw"
)

func main() {
sriov, err := ghw.SRIOV()
if err != nil {
fmt.Fprintf(os.Stderr, "Error getting SRIOV info: %v", err)
}

fmt.Printf("%v\n", sriov)

for _, dev := range sriov.PhysicalFunctions {
fmt.Printf(" %v\n", dev)
}
}
```

ghw discovers the SRIOV devices starting from the Physical Function (PF) and exposes them in the `PhysicalFunctions` slice.
Virtual Function (VF) are exposed as properties of the PF instance with exposes them.
However, in some cases users are interested in the VFs first, so it's clumsy to navigate the PFs to learn about VFs.
To make this easier, ghw also exposes a slice of VF instances. These instances are soft references to the very same VF objects
you can find from the PF objects.

```go
package main

import (
"fmt"

"github.com/jaypipes/ghw"
)

func main() {
sriov, err := ghw.SRIOV()
if err != nil {
fmt.Fprintf(os.Stderr, "Error getting SRIOV info: %v", err)
}

fmt.Printf("%v\n", sriov)

// you will see the very same VF data data you seen from the previous example
for _, dev := range sriov.VirtualFunctions {
fmt.Printf(" %v\n", dev)
}
}
```


### Chassis

The host's chassis information is accessible with the `ghw.Chassis()` function. This
Expand Down
9 changes: 9 additions & 0 deletions alias.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"github.com/jaypipes/ghw/pkg/pci"
pciaddress "github.com/jaypipes/ghw/pkg/pci/address"
"github.com/jaypipes/ghw/pkg/product"
"github.com/jaypipes/ghw/pkg/sriov"
"github.com/jaypipes/ghw/pkg/topology"
)

Expand Down Expand Up @@ -150,3 +151,11 @@ type GraphicsCard = gpu.GraphicsCard
var (
GPU = gpu.New
)

type SRIOVInfo = sriov.Info
type PhysicalFunction = sriov.PhysicalFunction
type VirtualFunction = sriov.VirtualFunction

var (
SRIOV = sriov.New
)
48 changes: 48 additions & 0 deletions cmd/ghwc/commands/sriov.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
//
// Use and distribution licensed under the Apache license version 2.
//
// See the COPYING file in the root project directory for full text.
//

package commands

import (
"fmt"

"github.com/jaypipes/ghw"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)

// sriovCmd represents the listing command
var sriovCmd = &cobra.Command{
Use: "sriov",
Short: "Show SRIOV devices information for the host system",
RunE: showSRIOV,
}

// showSRIOV show SRIOV physical device information for the host system.
func showSRIOV(cmd *cobra.Command, args []string) error {
sriov, err := ghw.SRIOV()
if err != nil {
return errors.Wrap(err, "error getting SRIOV info")
}

switch outputFormat {
case outputFormatHuman:
fmt.Printf("%v\n", sriov)

for _, dev := range sriov.PhysicalFunctions {
fmt.Printf(" %v\n", dev)
}
case outputFormatJSON:
fmt.Printf("%s\n", sriov.JSONString(pretty))
case outputFormatYAML:
fmt.Printf("%s", sriov.YAMLString())
}
return nil
}

func init() {
rootCmd.AddCommand(sriovCmd)
}
41 changes: 27 additions & 14 deletions host.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ package ghw

import (
"fmt"
"strings"

"github.com/jaypipes/ghw/pkg/context"

Expand All @@ -22,6 +23,7 @@ import (
"github.com/jaypipes/ghw/pkg/net"
"github.com/jaypipes/ghw/pkg/pci"
"github.com/jaypipes/ghw/pkg/product"
"github.com/jaypipes/ghw/pkg/sriov"
"github.com/jaypipes/ghw/pkg/topology"
)

Expand All @@ -40,6 +42,7 @@ type HostInfo struct {
Baseboard *baseboard.Info `json:"baseboard"`
Product *product.Info `json:"product"`
PCI *pci.Info `json:"pci"`
SRIOV *sriov.Info `json:"sriov"`
}

// Host returns a pointer to a HostInfo struct that contains fields with
Expand Down Expand Up @@ -91,6 +94,10 @@ func Host(opts ...*WithOption) (*HostInfo, error) {
if err != nil {
return nil, err
}
sriovInfo, err := sriov.New(opts...)
if err != nil {
return nil, err
}
return &HostInfo{
ctx: ctx,
CPU: cpuInfo,
Expand All @@ -104,26 +111,32 @@ func Host(opts ...*WithOption) (*HostInfo, error) {
Baseboard: baseboardInfo,
Product: productInfo,
PCI: pciInfo,
SRIOV: sriovInfo,
}, nil
}

// String returns a newline-separated output of the HostInfo's component
// structs' String-ified output
func (info *HostInfo) String() string {
return fmt.Sprintf(
"%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n",
info.Block.String(),
info.CPU.String(),
info.GPU.String(),
info.Memory.String(),
info.Network.String(),
info.Topology.String(),
info.Chassis.String(),
info.BIOS.String(),
info.Baseboard.String(),
info.Product.String(),
info.PCI.String(),
)
var b strings.Builder
for _, s := range []fmt.Stringer{
info.Block,
info.CPU,
info.GPU,
info.Memory,
info.Network,
info.Topology,
info.Chassis,
info.BIOS,
info.Baseboard,
info.Product,
info.PCI,
info.SRIOV,
} {
b.WriteString(s.String())
b.WriteString("\n")
}
return b.String()
}

// YAMLString returns a string with the host information formatted as YAML
Expand Down
10 changes: 10 additions & 0 deletions pkg/pci/address/address.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,16 @@ type Address struct {
Function string
}

func (addr *Address) Equal(a *Address) bool {
if addr == nil && a == nil {
return true
}
if addr != nil && a != nil {
return addr.Domain == a.Domain && addr.Bus == a.Bus && addr.Device == a.Device && addr.Function == a.Function
}
return false
}

// String() returns the canonical [D]BDF representation of this Address
func (addr *Address) String() string {
return addr.Domain + ":" + addr.Bus + ":" + addr.Device + "." + addr.Function
Expand Down
8 changes: 8 additions & 0 deletions pkg/pci/address/address_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,11 @@ func TestPCIAddressFromString(t *testing.T) {
}
}
}

func TestPCIAddressEqual(t *testing.T) {
addr1 := pciaddr.FromString("0000:03:00.A")
addr2 := pciaddr.FromString("03:00.A")
if addr1.Equal(addr2) == false {
t.Fatalf("addr1 %v and addr2 %v should be equal", addr1, addr2)
}
}
15 changes: 14 additions & 1 deletion pkg/snapshot/clonetree.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ func CopyFilesInto(fileSpecs []string, destDir string, opts *CopyFileOptions) er
if opts == nil {
opts = &CopyFileOptions{
IsSymlinkFn: isSymlink,
ShouldCreateDirFn: isDriversDir,
ShouldCreateDirFn: shouldCreateDir,
}
}
for _, fileSpec := range fileSpecs {
Expand Down Expand Up @@ -157,6 +157,13 @@ func copyFileTreeInto(paths []string, destDir string, opts *CopyFileOptions) err
return nil
}

func shouldCreateDir(path string, fi os.FileInfo) bool {
if isDeviceNetworkDir(path, fi) {
return true
}
return isDriversDir(path, fi)
}

func isSymlink(path string, fi os.FileInfo) bool {
return fi.Mode()&os.ModeSymlink != 0
}
Expand All @@ -165,6 +172,12 @@ func isDriversDir(path string, fi os.FileInfo) bool {
return strings.Contains(path, "drivers")
}

func isDeviceNetworkDir(path string, fi os.FileInfo) bool {
parentDir := filepath.Base(filepath.Dir(path))
// TODO: the "HasPrefix" check is brutal, but should work on linux
return parentDir == "net" && strings.HasPrefix(path, "/sys/devices")
}

func copyLink(path, targetPath string) error {
target, err := os.Readlink(path)
if err != nil {
Expand Down
48 changes: 48 additions & 0 deletions pkg/snapshot/clonetree_pci_linux.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,19 @@ func scanPCIDeviceRoot(root string) (fileSpecs []string, pciRoots []string) {
"revision",
"vendor",
}

perDevEntriesOpt := []string{
"driver",
"net/*",
"physfn",
"sriov_*",
"virtfn*",
}

ignoreSet := newKeySet(
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's necessary to create a new keySet typedef. We can just use map[string]bool here and where you're doing the ignoreSet.Contains call below, just do:

if _, ignored := ignoreSet[globbedEntry]; ignored {
     continue
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure thing, will simplify

"sriov_vf_msix_count", // linux >= 5.14, write-only
)

entries, err := ioutil.ReadDir(root)
if err != nil {
return []string{}, []string{}
Expand All @@ -96,6 +109,25 @@ func scanPCIDeviceRoot(root string) (fileSpecs []string, pciRoots []string) {
fileSpecs = append(fileSpecs, filepath.Join(pciEntry, perNetEntry))
}

for _, perNetEntryOpt := range perDevEntriesOpt {
netEntryOptPath := filepath.Join(pciEntry, perNetEntryOpt)

items, err := filepath.Glob(netEntryOptPath)
if err != nil {
// TODO: we skip silently because we don't have
// a ctx handy, so we can't do ctx.Warn :\
continue
}

for _, item := range items {
globbedEntry := filepath.Base(item)
if ignoreSet.Contains(globbedEntry) {
continue
}
fileSpecs = append(fileSpecs, item)
}
}

if isPCIBridge(entryPath) {
trace("adding new PCI root %q\n", entryName)
pciRoots = append(pciRoots, pciEntry)
Expand Down Expand Up @@ -149,3 +181,19 @@ func isPCIBridge(entryPath string) bool {
}
return false
}

// TODO: make this a real package
type keySet map[string]struct{}

func newKeySet(keys ...string) keySet {
ks := make(map[string]struct{})
for _, key := range keys {
ks[key] = struct{}{}
}
return ks
}

func (ks keySet) Contains(key string) bool {
_, ok := ks[key]
return ok
}
Loading