Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add module redeclaration check #4773

Closed

Conversation

SuperSodaSea
Copy link
Contributor

Add function modules_support.dependency_scanner.check_module_redeclaration to detect module interface / partition redeclaration before build.

Fixes #4770.

@SuperSodaSea SuperSodaSea marked this pull request as draft February 26, 2024 14:42
@SuperSodaSea
Copy link
Contributor Author

刚发现有个测试用来确保模块重复声明时的链接顺序(tests/projects/c++/modules/link_order)。也许应该加一个 flag 来开启重复定义的检查?不过我好像想不出有什么场景下模块重复声明是有用的。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I just found a test to ensure the link order when modules are repeatedly declared ([tests/projects/c++/modules/link_order](https://github.com/xmake-io/xmake/blob/76af94c04ba55dfe7a5087ce76dfeba4a709ba46/tests/projects/ c%2B%2B/modules/link_order/xmake.lua)). Maybe add a flag to enable duplicate definition checking? But I can't seem to think of any scenario where repeated module declarations would be useful.

@waruqi
Copy link
Member

waruqi commented Feb 26, 2024

我觉得这个不应该在 xmake 层面去检测,而是得让 linker 自己去检测和报错。。

既然你 issues 里面提到 cmake 能报错,说明它可能把两个 obj 都同时传给 linker 去 link 了,被 linker 检测出来了,而 xmake 里面可能仅仅是由于少传了一个 obj 给 linker ,导致没法被检测到,那可以改进下实现,让 xmake 把所有依赖的 obj 都传给 linker 才对,让 linker 去主动检测和报错。

B.obj : error LNK2005: "int __cdecl f(void)" (?f@@yahxz::<!A>) 已经在 A.obj 中定义
ModuleTest.exe: fatal error LNK1169: 找到一个或多个多重定义的符号

首先,xmake 内置的 dependency_scanner 实现,仅仅是 编译器没有原生支持时候的 fallback 方案,如果编译器支持,也是优先用的编译器的实现。 毕竟 xmake 提供的扫描实现方案,并不是完全可靠的。

另外,在 xmake 层面去检测这些,也会影响编译效率,尤其是工程变得庞大,模块依赖层次结构复杂的时候,这种检测是非常耗时的,lua 层来回遍历这些非常慢。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I think this should not be detected at the xmake level, but the linker must detect and report errors by itself. .

Since you mentioned in your issues that cmake can report an error, it means that it may have linked both of them at the same time and was detected by the linker. However, in xmake, it may just be that one less obj is passed to the linker, resulting in the failure to be detected. Then The implementation can be improved so that xmake passes all dependent objs to the linker.

B.obj: error LNK2005: "int __cdecl f(void)" (?f@@yahxz::<!A>) has been defined in A.obj
ModuleTest.exe: fatal error LNK1169: One or more multiply-defined symbols found

First of all, xmake's built-in dependency_scanner implementation is just a fallback solution when the compiler does not have native support. If the compiler supports it, the compiler's implementation will be used. After all, the scanning implementation solution provided by xmake is not completely reliable.

In addition, detecting these at the xmake level will also affect compilation efficiency. Especially when the project becomes large and the module dependency hierarchy is complex, this detection is very time-consuming.

@waruqi waruqi requested a review from Arthapz February 26, 2024 15:12
@waruqi
Copy link
Member

waruqi commented Feb 26, 2024

@Arthapz any idea?

@Arthapz
Copy link
Member

Arthapz commented Feb 26, 2024

@Arthapz any idea?

Oh sorry didn't saw the PR i'll Check very soon

@waruqi waruqi changed the base branch from master to dev February 26, 2024 22:47
@Arthapz
Copy link
Member

Arthapz commented Feb 27, 2024

so after some tests, we can enable the CMake behavior by deleting this line

which cull duplicate modules

and changing this one

local job_name = name and target:name() .. name or cppfile
to local job_name = cppfile
to actually build the duplicated module so the objectfile passed to the linker exists

but link_order test will not work anymore (and shall be removed as it doesn't test anything usefull ?)

> xmake -v
checking for platform ... linux
checking for architecture ... x86_64
checking for gcc ... /usr/bin/gcc
checking for zig ... ⭕
checking for zig ... ⭕
checking for gcc ... /usr/bin/gcc
checking for the c++ compiler (cxx) ... gcc
checking for /usr/bin/gcc ... ✔
checking for flags (-fPIC) ... ✔
checking for flags (gcc_modules_ts) ... ✔
checking for flags (gcc_deps_format) ... ⭕
checking for flags (gcc_deps_file) ... ⭕
checking for flags (gcc_deps_output) ... ⭕
[  0%]: <ModuleTest> generating.module.deps src/main.cpp
checking for flags (-std=c++20) ... ✔
checking for flags (-D_GLIBCXX_USE_CXX11_ABI=0) ... ✔
/usr/bin/gcc -E -x c++ -m64 -std=c++20 -D_GLIBCXX_USE_CXX11_ABI=0 src/main.cpp -o build/.gens/ModuleTest/linux/x86_64/release/rules/bmi/cache/modules/10b08683/main.cpp.i
[  0%]: <ModuleTest> generating.module.deps src/B.mpp
/usr/bin/gcc -E -x c++ -m64 -std=c++20 -D_GLIBCXX_USE_CXX11_ABI=0 src/B.mpp -o build/.gens/ModuleTest/linux/x86_64/release/rules/bmi/cache/modules/10b08683/B.mpp.i
[  0%]: <ModuleTest> generating.module.deps src/A.mpp
/usr/bin/gcc -E -x c++ -m64 -std=c++20 -D_GLIBCXX_USE_CXX11_ABI=0 src/A.mpp -o build/.gens/ModuleTest/linux/x86_64/release/rules/bmi/cache/modules/10b08683/A.mpp.i
checking for flags (gcc_module_mapper) ... ✔
[ 37%]: <ModuleTest> compiling.module.release A
/usr/bin/gcc -c -m64 -std=c++20 -fmodules-ts -D_GLIBCXX_USE_CXX11_ABI=0 -fmodule-mapper=/tmp/.xmake1000/240227/ModuleTest/src/A.mpp -x c++ -o build/.objs/ModuleTest/linux/x86_64/release/src/A.mpp.o sr
c/A.mpp
[ 37%]: <ModuleTest> compiling.module.release A
/usr/bin/gcc -c -m64 -std=c++20 -fmodules-ts -D_GLIBCXX_USE_CXX11_ABI=0 -fmodule-mapper=/tmp/.xmake1000/240227/ModuleTest/src/B.mpp -x c++ -o build/.objs/ModuleTest/linux/x86_64/release/src/B.mpp.o sr
c/B.mpp
checking for flags (-fdiagnostics-color=always) ... ✔
[ 62%]: compiling.release src/main.cpp
/usr/bin/gcc -c -m64 -std=c++20 -fmodules-ts -D_GLIBCXX_USE_CXX11_ABI=0 -fmodule-mapper=/tmp/.xmake1000/240227/ModuleTest/src/main.cpp -o build/.objs/ModuleTest/linux/x86_64/release/src/main.cpp.o src
/main.cpp
checking for flags (-MMD -MF) ... ✔
checking for g++ ... /usr/bin/g++
checking for the linker (ld) ... g++
checking for /usr/bin/g++ ... ✔
checking for flags (-fPIC) ... ✔
[ 75%]: linking.release ModuleTest
/usr/bin/g++ -o build/linux/x86_64/release/ModuleTest build/.objs/ModuleTest/linux/x86_64/release/src/main.cpp.o build/.objs/ModuleTest/linux/x86_64/release/src/A.mpp.o build/.objs/ModuleTest/linux/x8
6_64/release/src/B.mpp.o -m64
/usr/bin/ld: build/.objs/ModuleTest/linux/x86_64/release/src/B.mpp.o: in function `f@A()':
B.mpp:(.text+0x0): multiple definition of `f@A()'; build/.objs/ModuleTest/linux/x86_64/release/src/A.mpp.o:A.mpp:(.text+0x0): first defined here
/usr/bin/ld: build/.objs/ModuleTest/linux/x86_64/release/src/B.mpp.o: in function `initializer for module A':
B.mpp:(.text+0xb): multiple definition of `initializer for module A'; build/.objs/ModuleTest/linux/x86_64/release/src/A.mpp.o:A.mpp:(.text+0xb): first defined here
collect2: error: ld returned 1 exit status
❗ error: execv(/usr/bin/g++ -o build/linux/x86_64/release/ModuleTest build/.objs/ModuleTest/linux/x86_64/release/src/main.cpp.o build/.objs/ModuleTest/linux/x86_64/release/src/A.mpp.o build/.objs/Mod
uleTest/linux/x86_64/release/src/B.mpp.o -m64) failed(1)

@SuperSodaSea
Copy link
Contributor Author

@Arthapz Looks cool, do you think this could be the default behavior for xmake?

@Arthapz
Copy link
Member

Arthapz commented Feb 29, 2024

@Arthapz Looks cool, do you think this could be the default behavior for xmake?

sure, if @waruqi is okay with this, you can update ur PR with the modification hinted

EDIT: it may require some work on the should_build / mark_build functions

@waruqi
Copy link
Member

waruqi commented Mar 1, 2024

@Arthapz Looks cool, do you think this could be the default behavior for xmake?

sure, if @waruqi is okay with this, you can update ur PR with the modification hinted

EDIT: it may require some work on the should_build / mark_build functions

I'm ok.

@SuperSodaSea
Copy link
Contributor Author

Okay, I'll create a new PR for this.

@SuperSodaSea SuperSodaSea deleted the fix/module-redeclaration branch March 1, 2024 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

C++模块接口重复定义时会静默成功
4 participants