Repository navigation
Python layer 127+ logs "failed to enable ddtrace support for unittest" at ERROR on every init (ci_visibility stripped, unittest integration still enabled) #871
Description
Expected Behavior
Importing unittest (directly or through a dependency) in a function using the Datadog Python layer does not produce errors. Test-runner integrations should be inert in Lambda, since the layer deliberately removes CI Visibility.
Actual Behavior
Every cold start and provisioned-concurrency init logs this at ERROR:
[ERROR] failed to enable ddtrace support for unittest: No module named 'ddtrace.internal.ci_visibility'
Requests are unaffected, but the message trips error-log monitors. It started when we moved from layer 116 to 127. On the same function and code, ddtrace's startup line went from "Configured ddtrace instrumentation for 78 integration(s)" to 84.
Cause, from reading the layer Dockerfile on main and dd-trace-py v4.12.2 (the version layer 127 bundles for Python >= 3.10):
ddtrace/_monkey.pyhas"unittest": TrueinPATCH_MODULES, anddatadog_lambda/patch.pycallsddtrace.patch_all()without disabling it. So an import hook is registered forunittest.- The Dockerfile's "CI Visibility paths/integrations" step removes
ddtrace/internal/ci_visibility(added in chore: remove all ddtrace ci viz paths from layer #765). It also removesddtrace/contrib/unittest/, but in ddtrace 4.x the integration lives atddtrace/contrib/internal/unittest/. That directory survives, and itspatch.pyimportsddtrace.internal.ci_visibilityon line 29. - When
unittestis imported, the hook (_monkey.pyline 278) imports that patch module, which fails. The hook is registered withraise_errors=False, so the failure is logged at ERROR rather than raised.
Removing contrib/internal/unittest/ alone would not fix it: the hook would still fail to import the patch module and log the same ERROR. The integration needs to be disabled. For example, datadog_lambda could call patch_all_dd(unittest=False), or more generally turn off the test-runner integrations whose dependencies the layer strips. The same likely applies to pytest and pytest_bdd if they are in the default patch list, and their strip paths in the Dockerfile have the same pre-4.x layout. We have not checked those.
Layers 128 and 129 appear unchanged in this respect: nothing related is in their release notes, and the Dockerfile on main has the same strip list.
Steps to Reproduce the Problem
- Deploy a Python 3.11 function with
Datadog-Python311-ARM:127andDatadog-Extension-ARM:99, withDD_TRACE_ENABLED=true. - Have the handler module run
import unittestat module level (we see it through a dependency, not our own code). - Invoke it, then check the function's logs for
failed to enable ddtrace support for unittest.
Specifications
- Datadog Lambda Layer version: Python 127 (dd-trace-py 4.12.2), extension 99
- Python version: 3.11 (arm64)
Stacktrace
[ERROR] failed to enable ddtrace support for unittest: No module named 'ddtrace.internal.ci_visibility'
Traceback (most recent call last):
File "./python/lib/python3.11/site-packages/ddtrace/_monkey.py", line 278, in on_import
File "/var/lang/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "./python/lib/python3.11/site-packages/datadog_lambda/cold_start.py", line 130, in wrapped_method
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "./python/lib/python3.11/site-packages/ddtrace/contrib/internal/unittest/patch.py", line 29, in <module>
ModuleNotFoundError: No module named 'ddtrace.internal.ci_visibility'
Workaround: set DD_TRACE_UNITTEST_ENABLED=false on the function. _patch_all honours it; DD_PATCH_MODULES does not apply on the patch_all() path.
Activity
rithikanarayan commentedon Oct 21, 2026
Hi @bubirose, thanks for reporting this issue! We are currently addressing it in #873 and will release a new version of the Lambda layer with the fix once it is ready. I'll provide another update here once the new version of datadog-lambda is released. Feel free to let us know if you see any other issues, thanks!
rithikanarayan commentedon Oct 21, 2026
Hi @bubirose, v130 of the datadog-lambda library has been released with the fix! I tested it in a Python 3.11 ARM Lambda to ensure that there are no error logs being emitted, but please reach out if you have any more issues. Thanks!