Skip to content

Commit

Permalink
Add logging verbosity level flag to extract_ir.py (#278)
Browse files Browse the repository at this point in the history
* Add logging verbosity level flag to extract_ir.py

This commit adds a logging verbosity level flag to extract_ir.py and the
appropriate plumbing/implementation in extract_ir_lib.py. This is
primarily motivated by these errors coming up quite often in non-trivial
builds (as it is fairly often there is some assembly or some flags don't
get passed around somewhere) and not providing a very high signal to
noise ratio, especially when used as a library against a bunch of
projects at once.

* Address reviewer feedback

- Don't change logging verbosity back to info to print the status
  message if the verbosity has been set lower.

* Address reviewer feedback

- Log subprocess output directly rather than having subprocess write
  directly to STDOUT/STDERR.

* Address reviewer feedback

- Switch to using subprocess.check_output
  • Loading branch information
boomanaiden154 authored Jul 20, 2023
1 parent b476595 commit 45a45dd
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 4 deletions.
5 changes: 5 additions & 0 deletions compiler_opt/tools/extract_ir.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@
In a local ThinLTO case, the compilation is assumedto have been performed
specifying -Wl,--save-temps=import -Wl,--thinlto-emit-index-files
To change the logging verbosity, pass an integer representing the desired
verbosity to the --verbosity flag. Use 0 for all logs, status information,
and detailed debug information, -1 for solely warnings, and -2 to not produce
any output.
"""

import json
Expand Down
11 changes: 7 additions & 4 deletions compiler_opt/tools/extract_ir_lib.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,9 +135,10 @@ def _extract_clang_artifacts(self, llvm_objcopy_path: str, cmd_filter: str,
return None
os.makedirs(self.dest_dir(), exist_ok=True)
try:
subprocess.run(
subprocess.check_output(
self._get_extraction_cmd_command(llvm_objcopy_path, cmd_section_name),
check=True)
stderr=subprocess.STDOUT,
encoding='utf-8')
if cmd_filter is not None or is_thinlto:
with open(self.cmd_file(), encoding='utf-8') as f:
lines = f.readlines()
Expand All @@ -153,13 +154,15 @@ def _extract_clang_artifacts(self, llvm_objcopy_path: str, cmd_filter: str,
index_file = get_thinlto_index(cmdline, self.obj_base_dir())
shutil.copy(index_file, self.thinlto_index_file())

subprocess.run(
subprocess.check_output(
self._get_extraction_bc_command(llvm_objcopy_path,
bitcode_section_name),
check=True)
stderr=subprocess.STDOUT,
encoding='utf-8')
except subprocess.CalledProcessError as e:
# This may happen if .o file was build from asm (.S source).
logging.warning('%s was not processed: %s', self.input_obj(), e)
logging.info(e.output)
return None
assert (os.path.exists(self.cmd_file()) and
os.path.exists(self.bc_file()) and
Expand Down

0 comments on commit 45a45dd

Please sign in to comment.