Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC Commits: c635f164 by Ben Gamari at 2025-07-15T14:05:54-04:00 configure: Drop probing of ld.gold As noted in #25716, `gold` has been dropped from binutils-2.44. Fixes #25716. Metric Increase: size_hello_artifact_gzip size_hello_unicode_gzip ghc_prim_so - - - - - 637bb538 by Ben Gamari at 2025-07-15T14:05:55-04:00 testsuite/recomp015: Ignore stderr This is necessary since ld.bfd complains that we don't have a .note.GNU-stack section, potentially resulting in an executable stack. - - - - - d3cd4ec8 by Wen Kokke at 2025-07-15T14:06:39-04:00 Fix documentation for heap profile ID - - - - - 73082769 by Ben Gamari at 2025-07-15T16:56:38-04:00 Bump win32-tarballs to v0.9 - - - - - 3b63b254 by Ben Gamari at 2025-07-15T16:56:39-04:00 rts/LoadArchive: Handle null terminated string tables As of `llvm-ar` now emits filename tables terminated with null characters instead of the usual POSIX `/\n` sequence. Fixes #26150. - - - - - 195f6527 by Tamar Christina at 2025-07-15T16:56:39-04:00 rts: rename label so name doesn't conflict with param - - - - - 63373b95 by Tamar Christina at 2025-07-15T16:56:39-04:00 rts: Handle API set symbol versioning conflicts - - - - - 48e9aa3e by Tamar Christina at 2025-07-15T16:56:39-04:00 rts: Mark API set symbols as HIDDEN and correct symbol type - - - - - 959e827a by Tamar Christina at 2025-07-15T16:56:39-04:00 rts: Implement WEAK EXTERNAL undef redirection by target symbol name - - - - - 65f19293 by Ben Gamari at 2025-07-15T16:56:39-04:00 rts/LoadArchive: Handle string table entries terminated with / llvm-ar appears to terminate string table entries with `/\n` [1]. This matters in the case of thin archives, since the filename is used. In the past this worked since `llvm-ar` would produce archives with "small" filenames when possible. However, now it appears to always use the string table. [1] https://github.com/llvm/llvm-project/blob/bfb686bb5ba503e9386dc899e1ebbe2488... - - - - - 9cbb3ef5 by Ben Gamari at 2025-07-15T16:56:39-04:00 testsuite: Mark T12497 as fixed Thanks to the LLVM toolchain update. Closes #22694. - - - - - 2854407e by Ben Gamari at 2025-07-15T16:56:39-04:00 testsuite: Accept new output of T11223_link_order_a_b_2_fail on Windows The archive member number changed due to the fact that llvm-ar now uses a string table. - - - - - 28439593 by Ben Gamari at 2025-07-15T16:56:39-04:00 rts/linker/PEi386: Implement IMAGE_REL_AMD64_SECREL This appears to now be used by libc++ as distributed by msys2. - - - - - 2b053755 by Tamar Christina at 2025-07-15T16:56:39-04:00 rts: Cleanup merge resolution residue in lookupSymbolInDLL_PEi386 and make safe without dependent - - - - - ec92df61 by Wen Kokke at 2025-07-16T04:36:02-04:00 Remove the `profile_id` parameter from various RTS functions. Various RTS functions took a `profile_id` parameter, intended to be used to distinguish parallel heap profile breakdowns (e.g., `-hT` and `-hi`). However, this feature was never implemented and the `profile_id` parameter was set to 0 throughout the RTS. This commit removes the parameter but leaves the hardcoded profile ID in the functions that emit the encoded eventlog events as to not change the protocol. The affected functions are `traceHeapProfBegin`, `postHeapProfBegin`, `traceHeapProfSampleString`, `postHeapProfSampleString`, `traceHeapProfSampleCostCentre`, and `postHeapProfSampleCostCentre`. - - - - - 3049e37d by Wen Kokke at 2025-07-16T04:36:02-04:00 Make `traceHeapProfBegin` an init event. - - - - - 16 changed files: - docs/users_guide/eventlog-formats.rst - m4/find_ld.m4 - mk/get-win32-tarballs.py - rts/ProfHeap.c - rts/RetainerSet.c - rts/Trace.c - rts/Trace.h - rts/eventlog/EventLog.c - rts/eventlog/EventLog.h - rts/linker/LoadArchive.c - rts/linker/PEi386.c - testsuite/tests/driver/recomp015/all.T - testsuite/tests/rts/all.T - testsuite/tests/rts/linker/T11223/T11223_link_order_a_b_2_fail.stderr-ws-32-mingw32 - testsuite/tests/rts/linker/T11223/T11223_link_order_a_b_2_fail.stderr-ws-64-mingw32 - utils/ghc-toolchain/src/GHC/Toolchain/Tools/Link.hs Changes: ===================================== docs/users_guide/eventlog-formats.rst ===================================== @@ -693,6 +693,8 @@ A single fixed-width event emitted during program start-up describing the sample :field String: retainer filter :field String: biography filter +The profile ID field is reserved for future use. + Cost centre definitions ^^^^^^^^^^^^^^^^^^^^^^^ @@ -792,6 +794,7 @@ Otherwise, a :event-type:`HEAP_PROF_SAMPLE_STRING` event is emitted instead. :field Word8: stack depth :field Word32[]: cost centre stack starting with inner-most (cost centre numbers) +The profile ID field is reserved for future use. String break-down ^^^^^^^^^^^^^^^^^ @@ -818,6 +821,8 @@ If the heap profile type is set to :rts-flag:`-hc` or :rts-flag:`-hb`, a :event- :field Word64: heap residency in bytes :field String: sample label +The profile ID field is reserved for future use. + .. _time-profiler-events: Time profiler event log output ===================================== m4/find_ld.m4 ===================================== @@ -21,14 +21,7 @@ AC_DEFUN([FIND_LD],[ return fi - case $CPU in - i386) - # We refuse to use ld.gold on i386 due to #23579, which we don't - # have a good autoconf check for. - linkers="ld.lld ld" ;; - *) - linkers="ld.lld ld.gold ld" ;; - esac + linkers="ld.lld ld" # Manually iterate over possible names since we want to ensure that, e.g., # if ld.lld is installed but gcc doesn't support -fuse-ld=lld, that we ===================================== mk/get-win32-tarballs.py ===================================== @@ -8,7 +8,7 @@ import argparse import sys from sys import stderr -TARBALL_VERSION = '0.8' +TARBALL_VERSION = '0.9' BASE_URL = "https://downloads.haskell.org/ghc/mingw/{}".format(TARBALL_VERSION) DEST = Path('ghc-tarballs/mingw-w64') ARCHS = ['x86_64', 'sources'] ===================================== rts/ProfHeap.c ===================================== @@ -557,7 +557,7 @@ initHeapProfiling(void) restore_locale(); - traceHeapProfBegin(0); + traceInitEvent(traceHeapProfBegin); } void @@ -896,17 +896,17 @@ dumpCensus( Census *census ) // Eventlog - traceHeapProfSampleString(0, "VOID", + traceHeapProfSampleString("VOID", (census->void_total * sizeof(W_))); - traceHeapProfSampleString(0, "LAG", + traceHeapProfSampleString("LAG", ((census->not_used - census->void_total) * sizeof(W_))); - traceHeapProfSampleString(0, "USE", + traceHeapProfSampleString("USE", ((census->used - census->drag_total) * sizeof(W_))); - traceHeapProfSampleString(0, "INHERENT_USE", + traceHeapProfSampleString("INHERENT_USE", (census->prim * sizeof(W_))); - traceHeapProfSampleString(0, "DRAG", + traceHeapProfSampleString("DRAG", (census->drag_total * sizeof(W_))); traceHeapProfSampleEnd(era); @@ -941,33 +941,33 @@ dumpCensus( Census *census ) switch (RtsFlags.ProfFlags.doHeapProfile) { case HEAP_BY_CLOSURE_TYPE: fprintf(hp_file, "%s", (char *)ctr->identity); - traceHeapProfSampleString(0, (char *)ctr->identity, + traceHeapProfSampleString((char *)ctr->identity, count * sizeof(W_)); break; case HEAP_BY_INFO_TABLE: fprintf(hp_file, "%p", ctr->identity); char str[100]; sprintf(str, "%p", ctr->identity); - traceHeapProfSampleString(0, str, count * sizeof(W_)); + traceHeapProfSampleString(str, count * sizeof(W_)); break; #if defined(PROFILING) case HEAP_BY_CCS: fprint_ccs(hp_file, (CostCentreStack *)ctr->identity, RtsFlags.ProfFlags.ccsLength); - traceHeapProfSampleCostCentre(0, (CostCentreStack *)ctr->identity, + traceHeapProfSampleCostCentre((CostCentreStack *)ctr->identity, count * sizeof(W_)); break; case HEAP_BY_ERA: fprintf(hp_file, "%" FMT_Word, (StgWord)ctr->identity); char str_era[100]; sprintf(str_era, "%" FMT_Word, (StgWord)ctr->identity); - traceHeapProfSampleString(0, str_era, count * sizeof(W_)); + traceHeapProfSampleString(str_era, count * sizeof(W_)); break; case HEAP_BY_MOD: case HEAP_BY_DESCR: case HEAP_BY_TYPE: fprintf(hp_file, "%s", (char *)ctr->identity); - traceHeapProfSampleString(0, (char *)ctr->identity, + traceHeapProfSampleString((char *)ctr->identity, count * sizeof(W_)); break; case HEAP_BY_RETAINER: ===================================== rts/RetainerSet.c ===================================== @@ -238,7 +238,7 @@ printRetainerSetShort(FILE *f, RetainerSet *rs, W_ total_size, uint32_t max_leng } } fputs(tmp, f); - traceHeapProfSampleString(0, tmp, total_size); + traceHeapProfSampleString(tmp, total_size); } /* ----------------------------------------------------------------------------- ===================================== rts/Trace.c ===================================== @@ -647,10 +647,10 @@ void traceTaskDelete_ (Task *task) } } -void traceHeapProfBegin(StgWord8 profile_id) +void traceHeapProfBegin(void) { if (eventlog_enabled) { - postHeapProfBegin(profile_id); + postHeapProfBegin(); } } void traceHeapBioProfSampleBegin(StgInt era, StgWord64 time) @@ -674,11 +674,10 @@ void traceHeapProfSampleEnd(StgInt era) } } -void traceHeapProfSampleString(StgWord8 profile_id, - const char *label, StgWord residency) +void traceHeapProfSampleString(const char *label, StgWord residency) { if (eventlog_enabled) { - postHeapProfSampleString(profile_id, label, residency); + postHeapProfSampleString(label, residency); } } @@ -718,11 +717,10 @@ void traceHeapProfCostCentre(StgWord32 ccID, } // This one is for .hp samples -void traceHeapProfSampleCostCentre(StgWord8 profile_id, - CostCentreStack *stack, StgWord residency) +void traceHeapProfSampleCostCentre(CostCentreStack *stack, StgWord residency) { if (eventlog_enabled) { - postHeapProfSampleCostCentre(profile_id, stack, residency); + postHeapProfSampleCostCentre(stack, residency); } } ===================================== rts/Trace.h ===================================== @@ -303,20 +303,18 @@ void traceTaskMigrate_ (Task *task, void traceTaskDelete_ (Task *task); -void traceHeapProfBegin(StgWord8 profile_id); +void traceHeapProfBegin(void); void traceHeapProfSampleBegin(StgInt era); void traceHeapBioProfSampleBegin(StgInt era, StgWord64 time); void traceHeapProfSampleEnd(StgInt era); -void traceHeapProfSampleString(StgWord8 profile_id, - const char *label, StgWord residency); +void traceHeapProfSampleString(const char *label, StgWord residency); #if defined(PROFILING) void traceHeapProfCostCentre(StgWord32 ccID, const char *label, const char *module, const char *srcloc, StgBool is_caf); -void traceHeapProfSampleCostCentre(StgWord8 profile_id, - CostCentreStack *stack, StgWord residency); +void traceHeapProfSampleCostCentre(CostCentreStack *stack, StgWord residency); void traceProfSampleCostCentre(Capability *cap, CostCentreStack *stack, StgWord ticks); @@ -369,14 +367,14 @@ void flushTrace(void); #define traceTaskCreate_(taskID, cap) /* nothing */ #define traceTaskMigrate_(taskID, cap, new_cap) /* nothing */ #define traceTaskDelete_(taskID) /* nothing */ -#define traceHeapProfBegin(profile_id) /* nothing */ +#define traceHeapProfBegin() /* nothing */ #define traceHeapProfCostCentre(ccID, label, module, srcloc, is_caf) /* nothing */ #define traceIPE(ipe) /* nothing */ #define traceHeapProfSampleBegin(era) /* nothing */ #define traceHeapBioProfSampleBegin(era, time) /* nothing */ #define traceHeapProfSampleEnd(era) /* nothing */ -#define traceHeapProfSampleCostCentre(profile_id, stack, residency) /* nothing */ -#define traceHeapProfSampleString(profile_id, label, residency) /* nothing */ +#define traceHeapProfSampleCostCentre(stack, residency) /* nothing */ +#define traceHeapProfSampleString(label, residency) /* nothing */ #define traceConcMarkBegin() /* nothing */ #define traceConcMarkEnd(marked_obj_count) /* nothing */ ===================================== rts/eventlog/EventLog.c ===================================== @@ -95,6 +95,13 @@ bool eventlog_enabled; // protected by state_change_mutex to ensure * buffer size, EVENT_LOG_SIZE. We must ensure that no variable-length event * exceeds this limit. For this reason we impose maximum length limits on * fields which may have unbounded values. + * + * Note [Profile ID] + * ~~~~~~~~~~~~~~~~~ + * The profile ID field of eventlog entries is reserved for future use, + * with an eye towards supporting multiple parallel heap profiles. + * In the current RTS, the profile ID is hardcoded to 0. + * */ static const EventLogWriter *event_log_writer = NULL; @@ -1219,7 +1226,7 @@ static HeapProfBreakdown getHeapProfBreakdown(void) } } -void postHeapProfBegin(StgWord8 profile_id) +void postHeapProfBegin(void) { ACQUIRE_LOCK(&eventBufMutex); PROFILING_FLAGS *flags = &RtsFlags.ProfFlags; @@ -1244,7 +1251,8 @@ void postHeapProfBegin(StgWord8 profile_id) CHECK(!ensureRoomForVariableEvent(&eventBuf, len)); postEventHeader(&eventBuf, EVENT_HEAP_PROF_BEGIN); postPayloadSize(&eventBuf, len); - postWord8(&eventBuf, profile_id); + // See Note [Profile ID]. + postWord8(&eventBuf, 0); postWord64(&eventBuf, TimeToNS(flags->heapProfileInterval)); postWord32(&eventBuf, getHeapProfBreakdown()); postStringLen(&eventBuf, flags->modSelector, modSelector_len); @@ -1286,8 +1294,7 @@ void postHeapProfSampleEnd(StgInt era) RELEASE_LOCK(&eventBufMutex); } -void postHeapProfSampleString(StgWord8 profile_id, - const char *label, +void postHeapProfSampleString(const char *label, StgWord64 residency) { ACQUIRE_LOCK(&eventBufMutex); @@ -1296,7 +1303,8 @@ void postHeapProfSampleString(StgWord8 profile_id, CHECK(!ensureRoomForVariableEvent(&eventBuf, len)); postEventHeader(&eventBuf, EVENT_HEAP_PROF_SAMPLE_STRING); postPayloadSize(&eventBuf, len); - postWord8(&eventBuf, profile_id); + // See Note [Profile ID]. + postWord8(&eventBuf, 0); postWord64(&eventBuf, residency); postStringLen(&eventBuf, label, label_len); RELEASE_LOCK(&eventBufMutex); @@ -1325,8 +1333,7 @@ void postHeapProfCostCentre(StgWord32 ccID, RELEASE_LOCK(&eventBufMutex); } -void postHeapProfSampleCostCentre(StgWord8 profile_id, - CostCentreStack *stack, +void postHeapProfSampleCostCentre(CostCentreStack *stack, StgWord64 residency) { ACQUIRE_LOCK(&eventBufMutex); @@ -1340,7 +1347,8 @@ void postHeapProfSampleCostCentre(StgWord8 profile_id, CHECK(!ensureRoomForVariableEvent(&eventBuf, len)); postEventHeader(&eventBuf, EVENT_HEAP_PROF_SAMPLE_COST_CENTRE); postPayloadSize(&eventBuf, len); - postWord8(&eventBuf, profile_id); + // See Note [Profile ID]. + postWord8(&eventBuf, 0); postWord64(&eventBuf, residency); postWord8(&eventBuf, depth); for (ccs = stack; ===================================== rts/eventlog/EventLog.h ===================================== @@ -163,14 +163,13 @@ void postTaskMigrateEvent (EventTaskId taskId, void postTaskDeleteEvent (EventTaskId taskId); -void postHeapProfBegin(StgWord8 profile_id); +void postHeapProfBegin(void); void postHeapProfSampleBegin(StgInt era); void postHeapBioProfSampleBegin(StgInt era, StgWord64 time_ns); void postHeapProfSampleEnd(StgInt era); -void postHeapProfSampleString(StgWord8 profile_id, - const char *label, +void postHeapProfSampleString(const char *label, StgWord64 residency); #if defined(PROFILING) @@ -180,8 +179,7 @@ void postHeapProfCostCentre(StgWord32 ccID, const char *srcloc, StgBool is_caf); -void postHeapProfSampleCostCentre(StgWord8 profile_id, - CostCentreStack *stack, +void postHeapProfSampleCostCentre(CostCentreStack *stack, StgWord64 residency); void postProfSampleCostCentre(Capability *cap, ===================================== rts/linker/LoadArchive.c ===================================== @@ -223,21 +223,22 @@ lookupGNUArchiveIndex(int gnuFileIndexSize, char **fileName_, char* gnuFileIndex, pathchar* path, size_t* thisFileNameSize, size_t* fileNameSize) { - int n; char *fileName = *fileName_; if (isdigit(fileName[1])) { - int i; - for (n = 2; isdigit(fileName[n]); n++) - ; - - fileName[n] = '\0'; - n = atoi(fileName + 1); if (gnuFileIndex == NULL) { errorBelch("loadArchive: GNU-variant filename " "without an index while reading from `%" PATH_FMT "'", path); return false; } + + int n; + for (n = 2; isdigit(fileName[n]); n++) + ; + + char *end; + fileName[n] = '\0'; + n = strtol(fileName + 1, &end, 10); if (n < 0 || n > gnuFileIndexSize) { errorBelch("loadArchive: GNU-variant filename " "offset %d out of range [0..%d] " @@ -245,17 +246,27 @@ lookupGNUArchiveIndex(int gnuFileIndexSize, char **fileName_, n, gnuFileIndexSize, path); return false; } - if (n != 0 && gnuFileIndex[n - 1] != '\n') { + + // Check that the previous entry ends with the expected + // end-of-string delimiter. +#if defined(mingw32_HOST_OS) +#define IS_SYMBOL_DELIMITER(STR) (STR =='\n' || STR == '\0') +#else +#define IS_SYMBOL_DELIMITER(STR) (STR =='\n') +#endif + if (n != 0 && !IS_SYMBOL_DELIMITER(gnuFileIndex[n - 1])) { errorBelch("loadArchive: GNU-variant filename offset " "%d invalid (range [0..%d]) while reading " "filename from `%" PATH_FMT "'", n, gnuFileIndexSize, path); return false; } - for (i = n; gnuFileIndex[i] != '\n'; i++) + + int i; + for (i = n; !IS_SYMBOL_DELIMITER(gnuFileIndex[i]); i++) ; - size_t FileNameSize = i - n - 1; + size_t FileNameSize = i - n; if (FileNameSize >= *fileNameSize) { /* Double it to avoid potentially continually increasing it by 1 */ @@ -264,6 +275,13 @@ lookupGNUArchiveIndex(int gnuFileIndexSize, char **fileName_, "loadArchive(fileName)"); } memcpy(fileName, gnuFileIndex + n, FileNameSize); + + + /* llvm-ar terminates string table entries with `/\n`. */ + if (fileName[FileNameSize-1] == '/') { + FileNameSize--; + } + fileName[FileNameSize] = '\0'; *thisFileNameSize = FileNameSize; } ===================================== rts/linker/PEi386.c ===================================== @@ -342,6 +342,98 @@ Finally, we enter `ocResolve`, where we resolve relocations and and allocate jump islands (using the m32 allocator for backing storage) as necessary. + Note [Windows API Set] + ~~~~~~~~~~~~~~~~~~~~~~ + Windows has a concept called API Sets [1][2] which is intended to be Windows's + equivalent to glibc's symbolic versioning. It is also used to handle the API + surface difference between different device classes. e.g. the API might be + handled differently between a desktop and tablet. + + This is handled through two mechanisms: + + 1. Direct Forward: These use import libraries to manage to first level + redirection. So what used to be in ucrt.dll is now redirected based on + ucrt.lib. Every API now points to a possible different set of API sets + each following the API set contract: + + * The name must begin either with the string api- or ext-. + * Names that begin with api- represent APIs that exist on all Windows + editions that satisfy the API's version requirements. + * Names that begin with ext- represent APIs that may not exist on all + Windows editions. + * The name must end with the sequence l<n>-<n>-<n>, where n consists of + decimal digits. + * The body of the name can be alphanumeric characters, or dashes (-). + * The name is case insensitive. + + Here are some examples of API set contract names: + + - api-ms-win-core-ums-l1-1-0 + - ext-ms-win-com-ole32-l1-1-5 + - ext-ms-win-ntuser-window-l1-1-0 + - ext-ms-win-ntuser-window-l1-1-1 + + Forward references don't require anything special from the calling + application in that the Windows loader through "LoadLibrary" will + automatically load the right reference for you if given an API set + name including the ".dll" suffix. For example: + + INFO: DLL api-ms-win-eventing-provider-l1-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-apiquery-l1-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\ntdll.dll by API set + INFO: DLL api-ms-win-core-processthreads-l1-1-3.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-processthreads-l1-1-2.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-processthreads-l1-1-1.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-processthreads-l1-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-registry-l1-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-heap-l1-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-heap-l2-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-memory-l1-1-1.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-memory-l1-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-memory-l1-1-2.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + INFO: DLL api-ms-win-core-handle-l1-1-0.dll was redirected to C:\WINDOWS\SYSTEM32\kernelbase.dll by API set + + Which shows how the loader has redirected some of the references used + by ghci. + + Historically though we've treated shared libs lazily. We would load\ + the shared library, but not resolve the symbol immediately and wait until + the symbol is requested to iterate in order through the shared libraries. + + This assumes that you ever only had one version of a symbol. i.e. we had + an assumption that all exported symbols in different shared libraries + should be the same, because most of the time they come from re-exporting + from a base library. This is a bit of a weak assumption and doesn't hold + with API Sets. + + For that reason the loader now resolves symbols immediately, and because + we now resolve using BIND_NOW we must make sure that a symbol loaded + through an OC has precedent because the BIND_NOW refernce was not asked + for. For that reason we load the symbols for API sets with the + SYM_TYPE_DUP_DISCARD flag set. + + 2. Reverse forwarders: This is when the application has a direct reference + to the old name of an API. e.g. if GHC still used "msvcrt.dll" or + "ucrt.dll" we would have had to deal with this case. In this case the + loader intercepts the call and if it exists the dll is loaded. There is + an extra indirection as you go from foo.dll => api-ms-foo-1.dll => foo_imp.dll + + But if the API doesn't exist on the device it's resolved to a stub in the + API set that if called will result in an error should it be called [3]. + + This means that usages of GetProcAddress and LoadLibrary to check for the + existance of a function aren't safe, because they'll always succeed, but may + result in a pointer to the stub rather than the actual function. + + WHat does this mean for the RTS linker? Nothing. We don't have a fallback + for if the function doesn't exist. The RTS is merely just executing what + it was told to run. It's writers of libraries that have to be careful when + doing dlopen()/LoadLibrary. + + + [1] https://learn.microsoft.com/en-us/windows/win32/apiindex/windows-apisets + [2] https://mingwpy.github.io/ucrt.html#api-set-implementation + [3] https://learn.microsoft.com/en-us/windows/win32/apiindex/detect-api-set-avai... + */ #include "Rts.h" @@ -882,7 +974,7 @@ addDLL_PEi386( const pathchar *dll_name, HINSTANCE *loaded ) goto error; } } else { - goto loaded; /* We're done. DLL has been loaded. */ + goto loaded_ok; /* We're done. DLL has been loaded. */ } } } @@ -890,7 +982,7 @@ addDLL_PEi386( const pathchar *dll_name, HINSTANCE *loaded ) // We failed to load goto error; -loaded: +loaded_ok: addLoadedDll(&loaded_dll_cache, dll_name, instance); addDLLHandle(buf, instance); if (loaded) { @@ -1055,7 +1147,8 @@ bool checkAndLoadImportLibrary( pathchar* arch_name, char* member_name, FILE* f // We must call `addDLL_PEi386` directly rather than `addDLL` because `addDLL` // is now a wrapper around `loadNativeObj` which acquires a lock which we // already have here. - const char* result = addDLL_PEi386(dll, NULL); + HINSTANCE instance; + const char* result = addDLL_PEi386(dll, &instance); stgFree(image); @@ -1069,6 +1162,28 @@ bool checkAndLoadImportLibrary( pathchar* arch_name, char* member_name, FILE* f } stgFree(dll); + + // See Note [Windows API Set] + // We must immediately tie the symbol to the shared library. The easiest + // way is to load the symbol immediately. We already have all the + // information so might as well + SymbolAddr* sym = lookupSymbolInDLL_PEi386 (symbol, instance, dll, NULL); + + // Could be an import descriptor etc, skip if no symbol. + if (!sym) + return true; + + // The symbol must have been found, and we can add it to the RTS symbol table + IF_DEBUG(linker, debugBelch("checkAndLoadImportLibrary: resolved symbol %s to %p\n", symbol, sym)); + // Because the symbol has been loaded before we actually need it, if a + // stronger reference wants to add a duplicate we should discard this + // one to preserve link order. + SymType symType = SYM_TYPE_DUP_DISCARD | SYM_TYPE_HIDDEN; + symType |= hdr.Type == IMPORT_OBJECT_CODE ? SYM_TYPE_CODE : SYM_TYPE_DATA; + + if (!ghciInsertSymbolTable(dll, symhash, symbol, sym, false, symType, NULL)) + return false; + return true; } @@ -1198,7 +1313,7 @@ lookupSymbolInDLL_PEi386 ( const SymbolName* lbl, HINSTANCE instance, pathchar* it generates call *__imp_foo, and __imp_foo here has exactly the same semantics as in __imp_foo = GetProcAddress(..., "foo") */ - if (sym == NULL && strncmp (lbl, "__imp_", 6) == 0) { + if (sym == NULL && dependent && strncmp (lbl, "__imp_", 6) == 0) { sym = GetProcAddress(instance, lbl + 6); if (sym != NULL) { @@ -1214,12 +1329,6 @@ lookupSymbolInDLL_PEi386 ( const SymbolName* lbl, HINSTANCE instance, pathchar* } } - sym = GetProcAddress(instance, lbl); - if (sym != NULL) { - /*debugBelch("found %s in %s\n", lbl,dll_name);*/ - return sym; - } - return NULL; } @@ -1821,6 +1930,27 @@ ocGetNames_PEi386 ( ObjectCode* oc ) } if(NULL != targetSection) addr = (SymbolAddr*) ((size_t) targetSection->start + getSymValue(info, targetSym)); + else + { + // Do the symbol lookup based on name, this follows Microsoft's weak external's + // format 3 specifications. Example header generated: + // api-ms-win-crt-stdio-l1-1-0.dll: file format pe-x86-64 + // + // SYMBOL TABLE: + // [ 0](sec -1)(fl 0x00)(ty 0)(scl 3) (nx 0) 0x0000000000000000 @comp.id + // [ 1](sec -1)(fl 0x00)(ty 0)(scl 3) (nx 0) 0x0000000000000000 @feat.00 + // [ 2](sec 0)(fl 0x00)(ty 0)(scl 2) (nx 0) 0x0000000000000000 _write + // [ 3](sec 0)(fl 0x00)(ty 0)(scl 105) (nx 1) 0x0000000000000000 write + // AUX lnno 3 size 0x0 tagndx 2 + // + // https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#auxiliary-fo... + SymbolName *target_sname = get_sym_name (getSymShortName (info, targetSym), oc); + if (target_sname) + addr = lookupSymbol_PEi386 (target_sname, oc, &type); + + IF_DEBUG(linker, debugBelch("weak external symbol @ %s => %s resolved to %p\n", \ + sname, target_sname, addr)); + } } else if ( secNumber == IMAGE_SYM_UNDEFINED && symValue > 0) { /* This symbol isn't in any section at all, ie, global bss. @@ -2115,6 +2245,13 @@ ocResolve_PEi386 ( ObjectCode* oc ) *(uint64_t *)pP = S + A; break; } + case 11: /* IMAGE_REL_AMD64_SECREL (PE constant 11) */ + { + uint64_t offset = S - (uint64_t) section.start; + CHECK((uint32_t) offset == offset); + *(uint32_t *)pP = offset + A; + break; + } case 2: /* R_X86_64_32 (ELF constant 10) - IMAGE_REL_AMD64_ADDR32 (PE constant 2) */ case 3: /* IMAGE_REL_AMD64_ADDR32NB (PE constant 3) */ case 17: /* R_X86_64_32S ELF constant, no PE mapping. See note [ELF constant in PE file] */ ===================================== testsuite/tests/driver/recomp015/all.T ===================================== @@ -5,7 +5,11 @@ test('recomp015', # See ticket:11022#comment:7 unless(opsys('linux') or opsys('solaris2') or opsys('openbsd'), skip), when(arch('arm'), skip), - js_skip # JS backend doesn't support .s assembly files + js_skip, # JS backend doesn't support .s assembly files + + # the linker sometimes throws warnings since we don't + # generate a .note.GNU-stack section + ignore_stderr, ], makefile_test, []) ===================================== testsuite/tests/rts/all.T ===================================== @@ -426,9 +426,7 @@ test('T10296b', [only_ways(['threaded2'])], compile_and_run, ['']) test('numa001', [ extra_run_opts('8'), unless(unregisterised(), extra_ways(['debug_numa'])), req_ghc_with_threaded_rts ] , compile_and_run, ['']) -test('T12497', [ unless(opsys('mingw32'), skip), expect_broken(22694) - ], - makefile_test, ['T12497']) +test('T12497', unless(opsys('mingw32'), skip), makefile_test, ['T12497']) test('T13617', [ unless(opsys('mingw32'), skip)], makefile_test, ['T13617']) ===================================== testsuite/tests/rts/linker/T11223/T11223_link_order_a_b_2_fail.stderr-ws-32-mingw32 ===================================== @@ -3,7 +3,7 @@ GHC runtime linker: fatal error: I found a duplicate definition for symbol whilst processing object file E:\ghc-dev\msys64\home\Tamar\ghc\testsuite\tests\rts\T11223\T11223_link_order_a_b_2_fail.run\libfoo_link_lib_3.a The symbol was previously defined in - E:\ghc-dev\msys64\home\Tamar\ghc\testsuite\tests\rts\T11223\T11223_link_order_a_b_2_fail.run\libbar_link_lib_3.a(#2:bar_link_lib_3.o) + E:\ghc-dev\msys64\home\Tamar\ghc\testsuite\tests\rts\T11223\T11223_link_order_a_b_2_fail.run\libbar_link_lib_3.a(#3:bar_link_lib_3.o) This could be caused by: * Loading two different object files which export the same symbol * Specifying the same object file twice on the GHCi command line ===================================== testsuite/tests/rts/linker/T11223/T11223_link_order_a_b_2_fail.stderr-ws-64-mingw32 ===================================== @@ -3,7 +3,7 @@ GHC runtime linker: fatal error: I found a duplicate definition for symbol whilst processing object file E:\ghc-dev\msys64\home\Tamar\ghc\testsuite\tests\rts\T11223\T11223_link_order_a_b_2_fail.run\libfoo_link_lib_3.a The symbol was previously defined in - E:\ghc-dev\msys64\home\Tamar\ghc\testsuite\tests\rts\T11223\T11223_link_order_a_b_2_fail.run\libbar_link_lib_3.a(#2:bar_link_lib_3.o) + E:\ghc-dev\msys64\home\Tamar\ghc\testsuite\tests\rts\T11223\T11223_link_order_a_b_2_fail.run\libbar_link_lib_3.a(#3:bar_link_lib_3.o) This could be caused by: * Loading two different object files which export the same symbol * Specifying the same object file twice on the GHCi command line ===================================== utils/ghc-toolchain/src/GHC/Toolchain/Tools/Link.hs ===================================== @@ -91,7 +91,7 @@ findLinkFlags enableOverride cc ccLink -- executable exists before trying cc. do _ <- findProgram (linker ++ " linker") emptyProgOpt ["ld."++linker] prog <$ checkLinkWorks cc prog - | linker <- ["lld", "gold", "bfd"] + | linker <- ["lld", "bfd"] , let prog = over _prgFlags (++["-fuse-ld="++linker]) ccLink ] <|> (ccLink <$ checkLinkWorks cc ccLink) View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/86943ca2569120f5bdcdf259d83d5ee... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/86943ca2569120f5bdcdf259d83d5ee... You're receiving this email because of your account on gitlab.haskell.org.