We are happy to announce the availability of the latest development version of DTrace for Linux (2.0.0-1.14). This new version is based on BPF and other Linux kernel tracing features and is implemented entirely as a userspace application. It can be used for tracing on any Linux kernel that provides BPF based tracing and BTF type data, although (as mentioned below) improved functionality depends on two (optional) kernel patches. The functionality is close to being feature-complete in comparison with the kernel moduule based version of DTrace for Linux (version 1.2.1-1). Development continues in an incremental fashion to make the full feature set of DTrace available using existing kernel features. WHERE TO FIND IT? The new version of DTrace for Linux is available at: https://github.com/oracle/dtrace-utils/tree/2.0-branch-dev The main development branch for DTrace for Linux is at: https://github.com/oracle/dtrace-utils/tree/devel The most recent release tag is 2.0.0-1.14. FEATURES - Providers: + cpc: CPU Performacne Counter probes + dtrace: BEGIN, END, and ERROR probes + fbt: Function Boundary Tracing (FBT) probes + lockstat: Locking realted probes + pid: Userspace function boundary tracing + proc: Process lifecycle related probes + profile: Timer-based profile-* and tick-* probes + rawtp: SDT-style probes for kernel tracepoints with access to raw (untranslated) tracepoint arguments + sched: CPU scheduling probes [partial implementation] + sdt: Statically Defined Tracing (SDT) probes for kernel tracepoints + sycall: System call entry and exit probes + usdt: Userspace Statically Defined Tracing (USDT) probes + [NEW] pid: offset-based instruction probes - Aggregations: + Regular and indexed aggregations + Aggregation functions: avg, count, llquantize, lquantize, max, min, quantize, stddev, and sum. + Aggregation actions: clear, normalize, normalize, printa + [NEW] Support for using stack() and ustack() as aggregation keys - Speculative tracing: + Functions: speculation, speculate, commit, and discard - Variables: + Global variables + Thread-Local Storage (TLS) variables + Clause-local variables + Associative arrays for global and TLS variables + Full support for NULL-strings + Built-in: arg0 - arg9, args[], caller, curcpu, curthread, epid, errno, execname, gid, id, pid, ppid, probefunc, probemod, probename, probeprov, stackdepth, tid, timestamp, ucaller, uid, uregs[], ustackdepth, walltimestamp + [MEW] Elements in associative arrays of any type can now be deleted by assigning a literal 0. - Actions: + exit, freopen, ftruncate, mod, printa, printf, raise, setopt, stack, sym, system, trace, tracemem, uaddr, umod, ustack, usym + [NEW] print: output type-annotated structured data - Subroutines: + alloca, basename, bcopy, copyin, copyinstr, copyinto, copyout, copyoutstr, dirname, getmajor, getminor, htonl, htonll, htons, index, inet_ntoa, lltostr, mutex_owned, mutex_owner, mutex_type_adaptive, mutex_type_spin, ntohl, ntohll, ntohs, progenyof, rand, rindex, rw_iswriter, rw_read_held, rw_write_held, strchr, strjoin, strlen, strrchr, strstr, strtok, substr + [NEW] cleanpath() + [NEW] d_path(): dummy implementation to ensure translators compile + [NEW] link_ntop() - Runtime features: + Reporting of drop-counters for trace data that could not be recorded for the principal buffer, aggregation buffers, and speculation buffers. + [NEW] -xcpu: probe on a specific CPU only + [NEW] -xlockmem: set the locked memory limit to unlimited by default + [NEW] USDT data is persistent across dtprobed restarts. - BPF support: + Direct compilation of D source code into BPF programs. + Efficient use of pre-compiled BPF functions for library functions. + A bpflog option to request the BPF verifier log for loaded programs. + BPF program linking of dynamically generated code and pre-compiled code to facilitate code sharing and code re-use. + Improved integrated disassembler for generated BPF code at the clause and program level (-S in combination with the new -xdisasm=# option). + Improved trace data buffer handling based on memory mapped perf event ring-buffers. + [NEW] BTF type data support. - Development and debugging: + Support to run dtrace under valgrind. + [NEW] Configure script based building is supported. DEPENDENCIES DTrace for Linux depends on libctf (part of newer binutils) -or- libdtrace-ctf. While libctf is preferred, building against libdtrace-ctf is fully supported. It can be found at: https://github.com/oracle/libdtrace-ctf DTrace for Linux makes use of BPF library functions that are compiled at build time. It depends on BPF support in GCC and binutils to generate the pre-compiled BPF function library. DTrace for Linux makes use of 2 optional kernel features that are not commonly available in Linux kernels: - CTF data generation at compile time: this provides important datatype information for kernel and kernel module symbols. - Module symbol address range data: this adds address range data about any built-in modules to allow for consistent ways to refer to probes by module and function (or probe) name. DTrace for Linux can be used for tracing without these patches, albeit with some limitations. These additional support features for tracing are available at: https://github.com/oracle/dtrace-linux-kernel/tree/v2/6.7 Please consider joining our development list: dtrace-devel at oss.oracle.com and/or our IRC channel: #linux-dtrace at libera.chat Enjoy!