Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: enable AddressSanitizer #3026

Merged
merged 2 commits into from
Aug 1, 2024
Merged

ci: enable AddressSanitizer #3026

merged 2 commits into from
Aug 1, 2024

Conversation

Saviq
Copy link
Collaborator

@Saviq Saviq commented Aug 29, 2023

No description provided.

@codecov
Copy link

codecov bot commented Aug 29, 2023

Codecov Report

Merging #3026 (15d401c) into main (5a9c122) will decrease coverage by 0.11%.
Report is 8 commits behind head on main.
The diff coverage is n/a.

❗ Current head 15d401c differs from pull request most recent head 3b9fe2d. Consider uploading reports for the commit 3b9fe2d to get more accurate results

@@            Coverage Diff             @@
##             main    #3026      +/-   ##
==========================================
- Coverage   77.82%   77.71%   -0.11%     
==========================================
  Files        1064     1056       -8     
  Lines       74408    73394    -1014     
==========================================
- Hits        57906    57037     -869     
+ Misses      16502    16357     -145     

see 235 files with indirect coverage changes

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

@Saviq Saviq changed the base branch from main to cleanup-sanitizers August 30, 2023 15:21
@Saviq Saviq mentioned this pull request Aug 30, 2023
Base automatically changed from cleanup-sanitizers to main August 31, 2023 15:58
@mattkae
Copy link
Contributor

mattkae commented Jul 16, 2024

Runner.register_signal_handler_before_setup_invokes_callback_after_setup

I am 50% sure about this one, but I was able to fix it...

  • We are passing a std::initializer_list<int> to register_signal_handler by rvalue
  • We are most likely passing the values in this initializer list by reference, depending on the implementation
  • However, this is only a problem if we use it before we call miral::TestServer::SetUp(). This is because SetUp also registers a signal in mir::run_mir, which somehow results in a copy

I don't know why this all happens, but I will investigate deeper later

@mattkae
Copy link
Contributor

mattkae commented Jul 16, 2024

UdevWrapperDeathTest.DereferencingEndReturnsInvalidObject && UdevWrapperDeathTest.MemberDereferenceOfEndDies

In both cases, these are because of "The signal is caused by a READ memory access". I am 90% sure that the point of these tests is to have a bad memory access. @Saviq Is there a way to silence asan in these instances?

@mattkae
Copy link
Contributor

mattkae commented Jul 16, 2024

FullProbeStack.select_display_modules_loads_all_available_hardware_when_no_nested

  • This one is a simple case of loading stub_input_platform.cpp twice, leading the static members on StubInputPlatform to be defined twice.
  • This is because we link the executable to it and we load it at runtime
  • Simply not linking to $<TARGET_OBJECTS:mir-test-input-framework> would fix our issue, but InputPlatformProbe relies on having access to StubInputPlatform for one of its tests. Perhaps there's a clever away around this

@Saviq
Copy link
Collaborator Author

Saviq commented Jul 17, 2024

@Saviq Is there a way to silence asan in these instances?

This, I imagine:

https://github.com/google/sanitizers/wiki/AddressSanitizer#turning-off-instrumentation
https://clang.llvm.org/docs/AddressSanitizer.html#disabling-instrumentation-with-attribute-no-sanitize-address

@mattkae mattkae force-pushed the ci-enable-asan branch 3 times, most recently from bbdd1a4 to 4a5defa Compare July 17, 2024 20:07
@mattkae mattkae marked this pull request as ready for review July 18, 2024 13:18
@mattkae mattkae requested a review from a team as a code owner July 18, 2024 13:18
@mattkae
Copy link
Contributor

mattkae commented Jul 18, 2024

TextInputV2WithInputMethodV1Test.input_method_can_set_modifiers_map

That was a bug! The modifiers were being used after they were freed, since that resource isn't trivialyl copyable. It got released, and then we tried to use it later on in another thread

@mattkae
Copy link
Contributor

mattkae commented Jul 18, 2024

@AlanGriffiths Requesting you as a reviewer here since I modified some C++ stuff in a way that you might be interested in

@mattkae
Copy link
Contributor

mattkae commented Jul 18, 2024

Ah it's breaking again 🤔

@mattkae
Copy link
Contributor

mattkae commented Jul 18, 2024

FullProbeStack.* and InputPlatformProbe.*

These are failing (surprisingly only on clang) because mirplatforminputevdevobjects is getting loaded twice (once it is linked to at build time, the other time it is getting loaded dynamically).

Copy link
Contributor

@AlanGriffiths AlanGriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly, I don't like the input hub rework.

Classes that own resources (wl_array* in this case) should only manage that resource & data members should handle their own copy/assign semantics

src/include/server/mir/scene/text_input_hub.h Outdated Show resolved Hide resolved
Comment on lines 327 to 330
#if defined(__SANITIZE_ADDRESS__)
(void)udev_environment;
(void)KilledByInvalidMemoryAccess;
#else
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if silently passing the tests is the right pattern.

Having a list of tests excluded from running in sanitize builds seems better. (I think all "DeathTests" will be problematic - maybe just exclude them from the build?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think so too. I wonder if I could disable them in a smarter way globally

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Have a gtest_filter list to exclude from the run
  2. Move them to a separate file (or files) and don't build them

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For prior art see:

if(cmake_build_type_lower MATCHES "threadsanitizer")
if (NOT CMAKE_COMPILER_IS_GNUCXX)
find_program(LLVM_SYMBOLIZER llvm-symbolizer-3.6)
if (LLVM_SYMBOLIZER)
set(TSAN_EXTRA_OPTIONS "external_symbolizer_path=${LLVM_SYMBOLIZER}")
endif()
endif()
# Space after ${TSAN_EXTRA_OPTIONS} works around bug in TSAN env. variable parsing
list(APPEND test_env "TSAN_OPTIONS=\"suppressions=${CMAKE_SOURCE_DIR}/tools/tsan-suppressions second_deadlock_stack=1 halt_on_error=1 history_size=7 die_after_fork=0 ${TSAN_EXTRA_OPTIONS} \"")
# TSan does not support starting threads after fork
list(APPEND test_exclusion_filter "ThreadedDispatcherSignalTest.keeps_dispatching_after_signal_interruption")
# tsan "eats" SIGQUIT, so ignore two more tests that involve it
list(APPEND test_exclusion_filter "ServerSignal/AbortDeathTest.cleanup_handler_is_called_for/0")
list(APPEND test_exclusion_filter "ServerShutdown/OnSignalDeathTest.removes_endpoint/0")
endif()

tests/miral/runner.cpp Show resolved Hide resolved
Copy link
Contributor

@AlanGriffiths AlanGriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just simplifying CopyableWlArray

src/include/server/mir/scene/text_input_hub.h Outdated Show resolved Hide resolved
src/include/server/mir/scene/text_input_hub.h Outdated Show resolved Hide resolved
src/include/server/mir/scene/text_input_hub.h Outdated Show resolved Hide resolved
@mattkae
Copy link
Contributor

mattkae commented Jul 31, 2024

Trying to figure out why the asan_clang build works locally but fails in Github actions right now

@mattkae
Copy link
Contributor

mattkae commented Jul 31, 2024

Got the build error for clang asan:

[  7%] Building CXX object src/common/events/CMakeFiles/mirevents.dir/window_placement_event.cpp.o
cd /spread/mir/build-amd64/src/common/events && ccache /usr/bin/clang++ -DEGL_NO_X11 -DLTTNG_UST_HAVE_SDT_INTEGRATION -DMIR_LOG_COMPONENT_FALLBACK=\"mircommon\" -DMIR_USE_APPARMOR -DMIR_VERSION_MAJOR=2 -DMIR_VER
SION_MICRO=0 -DMIR_VERSION_MINOR=18 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/spread/mir/src/include/common -I/spread/mir/include/common -I/spread/mir/include/core -isystem /usr/include/libdrm -fpch-validate-input
-files-content -pthread -g -Wall -fno-strict-aliasing -pedantic -Wnon-virtual-dtor -Wextra -fPIC -Werror -Wno-mismatched-tags -Wno-psabi -std=c++23 -fsanitize=address -fno-omit-frame-pointer -MD -MT src/common/e
vents/CMakeFiles/mirevents.dir/window_placement_event.cpp.o -MF CMakeFiles/mirevents.dir/window_placement_event.cpp.o.d -o CMakeFiles/mirevents.dir/window_placement_event.cpp.o -c /spread/mir/src/common/events/w
indow_placement_event.cpp
make[2]: Leaving directory '/spread/mir/build-amd64'
`.text.asan.module_ctor.149' referenced in section `.init_array.1[asan.module_ctor.149]' of /tmp/lto-llvm-8df671.o: defined in discarded section `.text.asan.module_ctor.149[asan.module_ctor]' of /tmp/lto-llvm-8d
f671.o
`.text.asan.module_ctor.158' referenced in section `.init_array.1[asan.module_ctor.158]' of /tmp/lto-llvm-8df671.o: defined in discarded section `.text.asan.module_ctor.158[asan.module_ctor]' of /tmp/lto-llvm-8d
f671.o
`.text.asan.module_dtor.150' referenced in section `.fini_array.1[asan.module_dtor.150]' of /tmp/lto-llvm-8df671.o: defined in discarded section `.text.asan.module_dtor.150[asan.module_dtor]' of /tmp/lto-llvm-8d
f671.o
`.text.asan.module_dtor.159' referenced in section `.fini_array.1[asan.module_dtor.159]' of /tmp/lto-llvm-8df671.o: defined in discarded section `.text.asan.module_dtor.159[asan.module_dtor]' of /tmp/lto-llvm-8d
f671.o
[  7%] Built target wayland_generator_test_generated_files
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [examples/client/CMakeFiles/mir_demo_client_mir_shell.dir/build.make:118: bin/mir_demo_client_mir_shell.bin] Error 1
make[2]: Leaving directory '/spread/mir/build-amd64'
make[1]: *** [CMakeFiles/Makefile2:5044: examples/client/CMakeFiles/mir_demo_client_mir_shell.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/spread/mir/build-amd64'
[  7%] Built target mirevents
make[1]: Leaving directory '/spread/mir/build-amd64'
make: *** [Makefile:149: all] Error 2

@mattkae
Copy link
Contributor

mattkae commented Jul 31, 2024

We get many spurious failures in the asan build due to some threading issues, which isn't great...

@AlanGriffiths
Copy link
Contributor

We get many spurious failures in the asan build due to some threading issues, which isn't great...

If we identify threading issues in our code we should be fixing them

Comment on lines 102 to 133
class CopyableWlArray
{
public:
explicit CopyableWlArray(wl_array* in_data)
: data_{copy(in_data)}
{
}

CopyableWlArray(CopyableWlArray const& other)
: CopyableWlArray{other.data_}
{
}

CopyableWlArray& operator=(CopyableWlArray other)
{
std::swap(data_, other.data_);
return *this;
}

~CopyableWlArray()
{
wl_array_release(data_);
}

[[nodiscard]] wl_array* data() const
{
return data_;
}

private:
static wl_array* copy(wl_array* in_data)
{
auto data = new wl_array();
wl_array_init(data);
wl_array_copy(data, in_data);
return data;
}

wl_array* data_;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fails to delete data_, but the whole thing can be further simplfied...

Suggested change
class CopyableWlArray
{
public:
explicit CopyableWlArray(wl_array* in_data)
: data_{copy(in_data)}
{
}
CopyableWlArray(CopyableWlArray const& other)
: CopyableWlArray{other.data_}
{
}
CopyableWlArray& operator=(CopyableWlArray other)
{
std::swap(data_, other.data_);
return *this;
}
~CopyableWlArray()
{
wl_array_release(data_);
}
[[nodiscard]] wl_array* data() const
{
return data_;
}
private:
static wl_array* copy(wl_array* in_data)
{
auto data = new wl_array();
wl_array_init(data);
wl_array_copy(data, in_data);
return data;
}
wl_array* data_;
};
class CopyableWlArray
{
public:
explicit CopyableWlArray(wl_array const* in_data)
{
wl_array_init(&data_);
wl_array_copy(&data_, const_cast<wl_array*>(in_data)); // wl_array_copy() doesn't mark source as const
}
CopyableWlArray(CopyableWlArray const& other)
: CopyableWlArray{&other.data_}
{
}
CopyableWlArray& operator=(CopyableWlArray other)
{
std::swap(data_, other.data_);
return *this;
}
~CopyableWlArray()
{
wl_array_release(&data_);
}
[[nodiscard]] wl_array* data() const
{
// Our generated `send_modifiers_map_event()` requires a non-const wl_array, so
// this workaround may be the wrong thing
return const_cast<wl_array*>(&data_);
}
private:
wl_array data_;
};

tests/unit-tests/test_udev_wrapper.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@AlanGriffiths AlanGriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure about the StubInputPlatform rework.

The runner.cpp change is weird: the same incantation occurs in two other places and they are not changed.

But nothing I'll block on.

@AlanGriffiths
Copy link
Contributor

@mattkae I assume you are still investigating the ubuntu:asan_clang failure?

@mattkae
Copy link
Contributor

mattkae commented Aug 1, 2024

@mattkae I assume you are still investigating the ubuntu:asan_clang failure?

Yup. Doesn't repro on my machine, testing some things out now

@mattkae
Copy link
Contributor

mattkae commented Aug 1, 2024

Fixed & Cleaned up the commits. Just waiting on one last check 🤞

@mattkae mattkae enabled auto-merge August 1, 2024 15:30
@mattkae mattkae added this pull request to the merge queue Aug 1, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 1, 2024
@mattkae mattkae added this pull request to the merge queue Aug 1, 2024
@mattkae mattkae removed this pull request from the merge queue due to a manual request Aug 1, 2024
@mattkae mattkae added this pull request to the merge queue Aug 1, 2024
Merged via the queue into main with commit 75e3ca6 Aug 1, 2024
29 of 37 checks passed
@mattkae mattkae deleted the ci-enable-asan branch August 1, 2024 18:11
@AlanGriffiths AlanGriffiths mentioned this pull request Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants