DFReader: Create DFMetaData class to handle logger metadata #929
Conversation
shancock884
force-pushed
the
logmessage-help
branch
from
ee5b92d
to
d8f9988
Compare
There was a problem hiding this comment.
Have you checked how much extra RAM this consumes when you load an ArduPlane logfile? Is it simply roughly equal to the size of the metadata file, or does it blow out?
Also, what's the parse-time like on the file? "not noticable" is probably the answer, but a quick check with a pair of time.time() calls would be nice.
|
On Wed, 20 Mar 2024, Simon Hancock wrote:
@shancock884 commented on this pull request.
_______________________________________________________________________________________________________________________________________________________________________________________________________________
In DFReader.py:
> + def dot_pymavlink(self, *args):
+ '''return a path to store mavproxy data'''
+ if 'HOME' not in os.environ:
+ dir = os.path.join(os.environ['LOCALAPPDATA'], '.pymavlink')
+ else:
+ dir = os.path.join(os.environ['HOME'], '.pymavlink')
+ if len(args) == 0:
+ return dir
+ return os.path.join(dir, *args)
@peterbarker - It feels like this should be in mavutil or something for consistent use by all pymavlink functions.
(for example one place that accesses .mavproxy specifies "~" - which looks very Linux specific!)
Would you agree, and if so, shall I do so on this PR, or on a pre-PR, or a later PR?
I'd suggest keeping this self-contained initially and we can look at
factorying out on a future PR.
|
DFMetaData added as metadata attribute of DFReader class download static method to download XML into .pymavlink/LogMessages When needed, reads <vehicle>.xml file from ~/.pymavlink/LogMessages Method for printing help text for MAVExplorer 'logmessage help' command Method for returning log message description Applied code update suggestions and performed flake8 check on new code Co-authored-by: Peter Barker <pb-gh@barker.dropbear.id.au>
shancock884
force-pushed
the
logmessage-help
branch
from
aebb097
to
596855d
Compare
I still need to do the RAM/CPU time check, but I think all other comments are addressed. |
| @staticmethod | ||
| def download_url(url): | ||
| '''download a URL and return the content''' | ||
| if sys.version_info.major < 3: |
There was a problem hiding this comment.
Probably don't need this compatability code. IIRC the "exists_ok" used below makes this stuff Py3 only anyway...
| with gzip.GzipFile(fileobj=io.BytesIO(data)) as gz: | ||
| data = gz.read() | ||
| try: | ||
| open(file, mode='wb').write(data) |
There was a problem hiding this comment.
I've become rather partial to pathlib myself, but I do understand this is just swiped from MAVProxy.
| resp = url_open(url) | ||
| except url_error as e: | ||
| print('Error downloading %s : %s' % (url, e)) | ||
| return None |
There was a problem hiding this comment.
I wonder if we really should be doing this rather than making the caller catch the exceptions...
| if data is None: | ||
| continue | ||
| # decompress it... | ||
| with gzip.GzipFile(fileobj=io.BytesIO(data)) as gz: |
There was a problem hiding this comment.
We actually also produce .xz for these... they're about 75% the size of the .gz
|
Tested it. Merged,thanks! |
This PR adds a DFMetaData class to the DFReader.m file, which is used to handle LogMessage XML meta-data for use by MAVExplorer and potentially other tools. It links to PR ArduPilot/MAVProxy#1337.
The class is set as the metadata attribute of DFReader class, and at present includes methods:
Tested using the "logmessage help" command in MAVExplorer (with PR ArduPilot/MAVProxy#1337). For example: