Best practices for using this library

[erlend-aasland]

Hello! I'm looking for best practices for using this library (and implicitly also CAN/CANopen best practices, as I'm a completely newbie with this protocol). So far I've written down the following:

• after creating and connecting to a CAN network (network.connect()), call network.check()
• after the "device state" (NMT state), call state = node.nmt.wait_for_heartbeat() and check the result. Perhaps this should be a repeated process? (Call it X times before giving up?)
• after setting TPDO variables and calling .transmit(), call .wait_for_reception(); not sure about this one

If there are more obvious stuff, do's and dont's, or doh's, please shout out; I'm eager to learn :)

[sveinse]

Are you familiar with the CANopen protocol concepts in general? I'm wildly guessing what you know about CANopen, so pardon me if this is way too basic: I usually refer this to our new employees: https://www.csselectronics.com/pages/canopen-tutorial-simple-intro

The NMT service (Network Management) state is a "runmode" state that all CANopen devices shall have. When a device boots it shall start in PRE-OPERATIONAL and each device must be set into OPERATIONAL mode by a special agent on the network named NMT master. If you only do node.nmt.wait_for_heartbeat() you rely on that there is another node on the bus that sets the node into OPERATIONAL mode. If not, you can do this yourself from canopen by using node.nmt.set_state("OPERATIONAL").

PDO can only be generated in OPERATIONAL. SDOs are possible in both PRE-OPERATIONAL and OPERATIONAL mode. .wait_for_reception() is used for waiting on an incoming TPDO while .transmit() is for sending RPDO. Both of these must be setup up front in canopen, which is done with node.tpdo.read(from_od: bool) and node.rpdo.read(from_od: bool). The from_od argument controls if the PDO setup is retrieved from the local OD, or if it is fetched from the node using SDO requests.

(I'm sorry I don't have any readily off-hand examples, as we only use the asyncio port of canopen. I'd onlyt be misleading for the syntax.)

[erlend-aasland]

Thanks for the insights! I'll comment on the stuff I did not know beforehand posting :)

Are you familiar with the CANopen protocol concepts in general?

Just the basic concepts; started looking at CANopen a few weeks ago, so I'm pretty green at it 🍏 In the past, I've only worked with Modbus; CANopen is a different beast with its multiple communication modes.

I usually refer this to our new employees: https://www.csselectronics.com/pages/canopen-tutorial-simple-intro

Great, thanks for the link! 🙏

The NMT service (Network Management) state is a "runmode" state that all CANopen devices shall have. When a device boots it shall start in PRE-OPERATIONAL and each device must be set into OPERATIONAL mode by a special agent on the network named NMT master. If you only do node.nmt.wait_for_heartbeat() you rely on that there is another node on the bus that sets the node into OPERATIONAL mode. If not, you can do this yourself from canopen by using node.nmt.set_state("OPERATIONAL").

Yeah, I think I've got a good grasp of the NMT protocol. I guess good practice would be to configure NMT messages to be sent at a known interval? Currently, I'm relying on the default "stream" rate configured on the device¹.

SDOs are possible in both PRE-OPERATIONAL and OPERATIONAL mode.

But they have to be configured in PRE-OPERATIONAL, right?

The from_od argument controls if the PDO setup is retrieved from the local OD, or if it is fetched from the node using SDO requests.

Aha; I had my mental model configured the opposite way! 🙃

[...] we only use the asyncio port of canopen.

This is interesting; I'm currently using multi-threading, which seems to work fine with python-can and canopen, so async is not something I'm considering, but it's nice to know it's there. I'll check it out.

Footnotes

1. not sure if that's a correct assumption, though ↩

[erlend-aasland]

In the past, I've only worked with Modbus [...]

Just learned that CANopen can be implemented using Modbus as a transport layer 🤯

[acolomb]

Where did you get that advice? I think it's a bit misleading. CANopen depends on many properties of the CAN protocol, such as arbitration (for making sure the most important message gets through first), multi-master operation (in principle, each CAN node has equal bus access), and remote transfer requests. What some device manufacturers do is mapping the same Object Dictionary to be accessible via both CANopen and Modbus, by using the same index / subindex addressing. But that doesn't mean it's talking CANopen and just switching out the transport layer.

But maybe I just haven't seen such a crossover. How would it implement PDOs for example, with a "slave" device transmitting on its own without a request from the Modbus master?

[erlend-aasland]

André:

Where did you get that advice? I think it's a bit misleading.

I would not call it advice, but from the link that Svein provided, I quote:

It's worth noting, that CANopen could also be adapted to other data link layer protocols than CAN (e.g. EtherCAT, Modbus, Powerlink).

CANOpen on top of Modbus sounds like a disaster to me.

How would it implement PDOs for example, with a "slave" device transmitting on its own without a request from the Modbus master?

Exactly! Ditto for heartbeat/life guarding.

[acolomb]

• I've never used (and didn't know about) network.check(). Must be optional obviously :-) Usually better to put logging inside any callbacks you install and try to make them not raise exceptions.

• The heartbeat protocol is used for a number of different schemes, like Node Guarding, Life Guarding, and the Boot-up Protocol. wait_for_heartbeat() is only the correct approach if you know that one will be sent out by a device. That happens only once during power-on, or if you send the NMT state change command RESET or RESET COMMUNICATION. It's possible to send it as a broadcast by using network.nmt.state = 'RESET', if that is appropriate for your application lifecycle. I've had projects where the library is used as the master, putting other nodes into the correct state, but also where it's just listening and displaying values that are on the bus anyway, or requested via SDO. There is a standard 0x1017 object "Producer Heartbeat Time" that you can set on nodes to make them send a heartbeat (with current NMT state) regularly. I like to enable that on the "slave" nodes to easily tell when they are available before going OPERATIONAL. Otherwise, to detect a sign of life from a RemoteNode, receiving an associated TPDO implicitly shows it as alive (only in OPERATIONAL), and making a "confirmed service" request (e.g. SDO) and getting an answer is a good proof as well. Note that in NMT state STOPPED (which might not be announced without a heartbeat) your only change to "discover" the device is to command it to RESET COMMUNICATION and wait for another boot-up message (same as heartbeat). Because SDOs don't exist in STOPPED.

• PDO communication is usually coupled with the SYNC service, especially if you have multiple remote nodes and want a reproducible communication cycle. The SYNC master (which could be any device on the network, also canopen in Python), sends a very short indicator message. This triggers RPDOs received in remote nodes to actually update the used live values and at the same time take a "snapshot" of objects in TPDOs to be sent back following the SYNC. This goes well with the input - process - output cycle common in PLC systems, where the SYNC marks the boundary between cycles. This is the synchronous PDO transmission / reception mode, but there are others. The device may also send a TPDO whenever a value changes, without a SYNC cycle. And it may accept RPDO data immediately when received. These modes can be configured per PDO. If you know what kind of communication cycle you want to design, then it's rather easy mapping it onto the canopen API, but first you'll need to flesh out which modes are best suited for your application.

Note that one of the most beautiful details in CANopen protocol design is the possibility to remap PDOs. Not only their content, but also their applied COB-ID. This way, data that is already transmitted on the bus can be consumed by several nodes, without a master needing to receive from one and relay to another device. Just setting an RPDO's COB-ID to the value of another node's TPDO COB-ID will make the former node receive data directly from the latter, without additional delay of one cycle, and reducing the overall bus load. So you see the protocol is very flexible and API usage mostly depends on your desired communication structure. If all you need is scripted access to some parameters (not a cyclic control loop), then SDO is usually the right tool.

One important detail about SDO: Each "channel" (e.g. the default (0x580 / 0x600) + $NODEID pair for SDO1) is stateful, both on client and server side. That means if several clients want to exchange data with the same server, you either need to use several SDO channels (which many devices don't have). Or you need to coordinate which SDO client is allowed to use the channel by other means. For one application, we have this convention for example: The "master" device (central brain of the machine) configures all slaves (e.g. drive controllers, sensors) using SDO, after resetting them and waiting for their PRE-OPERATIONAL boot-up message. It then puts all devices in OPERATIONAL state, then ceases SDO communication and uses only SYNC+PDO (and the enabled heartbeat). Another node is used for diagnosis, but will only do SDO requests when the device is found to be in OPERATIONAL already, avoiding clashes with the master's SDO usage.

Hope some of this makes sense and gives you a better picture. We might be able to give more targeted advice if you describe the application where you want to employ the canopen API. Who is doing what, controlling whom, and which roles are filled by a node implemented in Python?

[erlend-aasland]

I've now rewritten and tested the app I've been handed using the best practices provided by you both, André and Svein, and I'm getting very satisfactory results. (For the last bits of optimisation, we need to make some hardware adjustments, but so far so good.) I've optimised the PDO mapping and I'm now making good use of the EMCY, Heart Beat, and Life Guarding protocols. Becoming more and more in love with CANOpen!

Once again, thank you so much for you insights, and a big thanks for maintaining canopen.

[acolomb]

The reason is that CAN only have one 11-bit arbitration ID, called COB-ID (Communication Object Identifier). This ID is further decoded into functions, such as SDO, PDO, SYNC, NMT, etc. Many of these include Node ID, which is the device identifier - the 11-bit encoding scheme is the reason there is "only" 127 addresses available in CANopen. Opposed to what Ethernet / IP does, 11-bit can does NOT specify source and destination address, it only specifies one of them. The effect of this is that stateful SDO traffic cannot happen in a one to many or many to one configuration without some arbitration.

It's actually a little more complicated. The "addresses" are not specific to a node. Of course, there is the "Predefined Connection Set" which has standard COB-IDs for each device based on its Node ID. But they can be changed freely if the device supports it. That applies to PDOs in both directions, but also for SDO there is a standard "channel" (0x580 / 0x600 + $NODEID) but more can be configured if the device has the capacity. Their COB-IDs can also be freely changed. So an SDO channel is definitely not bound to a certain device (Node ID). For the server side it makes sense to only have one device responding to SDO requests. But on the client side, any CAN participant can issue requests for the SDO channel. Ideally, all other clients that want to use the same channel, will see the request and update their internal client state to mirror that of the client who generated the SDO request. Remember that both request and response are always visible to all CAN participants, so e.g. uploading a value from a device can change the locally cached value in any other device, not only the one which made the request. If that principle is honored (with or without listening to "foreign request" responses), there should never be a clash because all nodes always know what state the SDO channel is in, and can wait until it's free.

I have experienced that SDO exchange with nodes that results in a one-package response usually works fine, but multi package responses makes the master confused and is shut down. Strictly speaking, both are protocol violations.

I don't quite get where you see a protocol violation here? Yes, sending multiple SDO requests to the same SDO channel (server) without waiting for the previous transfer to complete (or abort) is a protocol violation and leads to aborting or sometimes just missing responses if the devices are lazy. But it doesn't sound like you're actually using multiple SDO clients in parallel.

OTOH, SDO access is strictly separated from other protocols because of the arbitration ID, on the lower CAN layer. The different services are numbered in a way that prioritizes important messages, e.g. NMT < EMCY+SYNC < TIME < PDO < SDO < Guard. Lower numbers win during arbitration because of the 0 bits being dominant. If a CAN controller detects that its package transmission lost the arbitration, it backs off and retries when the other frame has finished. That's actually a brilliant way to avoid collisions without requiring the higher-priority sender to retry. It also guarantees a single source of truth regarding SDO protocol state. Because each node sees the same bits on the bus, they can all deduce a common state from the encountered transmissions. Unfortunately, not many CANopen stacks do update their SDO client state based on what requests are received. Instead they assume the client (themself) is the only possible source of requests, and therefore happily crash into other clients' running transactions. Using multiple SDO channels is a possible workaround of course. Note that the priority chain is also one of the few reasons for changing an SDO channel's COB-ID, to raise its priority above other protocols.

Summing up, the "SDO channel" is a stateful concept with shared state among any interested CANopen node. The bus access layer makes sure the state is unambiguous, but many implementations (including this library) don't expect and therefore don't respect state changes not initiated by themselves. That's also why the standard always talks about the "existance" of "objects" on the bus, instead of "sending" of "packets". The CAN bus has a single, globally known state with priority-based, collision-free MAC and that's a vast difference from other protocols such as EtherCAT. Those actually receive frames, mangle them, and retransmit to the next node, and there is no electrically parallel connection as in CAN. That's a latency nightmare, but doesn't matter too much because of the principally higher bit-rates.

[erlend-aasland]

The different services are numbered in a way that prioritizes important messages [...]. Lower numbers win during arbitration because of the 0 bits being dominant.

It just occurred to me that when designing the CANopen application, one should make sure the most important TPDO "stream" is using TPDO1, and the least important TPDO4. I already took care to make sure the most frequently transmitted PDO is as compact as possible, but since it is also the most important PDO, I should make sure it's using a low number.

TIL; thanks again for the insight!

[acolomb]

Prioritizing CAN objects really only matters when the bus is heavily loaded. In the most common real-time bus usage pattern, you have a SYNC cycle and the order of PDO transmissions before the next SYNC is not really important. Just that all PDOs should be transferred before then.

The reason why SYNC is usually preferred is simple, to avoid the cycle intervals to slowly run out of phase when using several devices on the bus. If each has its own transmission timer, inaccuracies will lead to PDO order changing eventually.

[sveinse]

Holy crap, you two have been prolific the last couple of days with all the activity 🥳 💪

I have experienced that SDO exchange with nodes that results in a one-package response usually works fine, but multi package responses makes the master confused and is shut down. Strictly speaking, both are protocol violations.

I don't quite get where you see a protocol violation here? Yes, sending multiple SDO requests to the same SDO channel (server) without waiting for the previous transfer to complete (or abort) is a protocol violation and leads to aborting or sometimes just missing responses if the devices are lazy. But it doesn't sound like you're actually using multiple SDO clients in parallel.

Our system consists of nodes assigned a single SDO channel 0x580 / 0x600 + $NODEID. It consists of a CANopen master that have active SDO communication to every node, even in OPERATIONAL. When I connect to this system using this library, I do that as an additional agent on the CAN-bus. What I observe is that if I use the SDO channel to the nodes and do SDO requests that results in single responses from the node, they come through fine. What fails thou, is SDO block transfers. My hypothesis is that the listening CANopen master is getting confused by the multi-message block sequence the bus and shuts it down. I haven't traced exactly what happens on the bus thou. It could be a result of how canfestival is behaving in SDO block transfer.

While discussing properties of CANopen I think there is another concept to be mentioned: There are in total 512 RPDOs and 512 TPDOs available in the system. By default PDOs are assigned COB-IDs which are related to $NODEID, 4 RPDOs and 4 TPDOs each. However, PDOs can be assigned freely, and not just limited to the default 4+4. Any node may thus have as many PDOs as necessary. What must be observed is that there is only one issuer of any given COB-ID and the default using $NODEID is a simple way to avoid that.