digitalmars.D.learn - Issue eBPF kernel programs with ldc?
- data pulverizer (280/280) Jan 05 # Issue eBPF kernel programs with ldc?
- Richard (Rikki) Andrew Cattermole (9/11) Jan 05 Compare your D definition against:
- data pulverizer (70/81) Jan 05 I have updated the kernel code to:
- Richard (Rikki) Andrew Cattermole (6/8) Jan 05 I don't understand why you gave a type declaration a section.
- data pulverizer (18/26) Jan 06 I don't actually think it matters what I do with the struct it
- Richard (Rikki) Andrew Cattermole (3/19) Jan 06 The only solution I can think of is to use pragma mangle on the struct,
I have been trying to get eBPF kernel programs running with D on
the LDC compiler. The object file generated by the LDC compiler
fails when attempting to run it with the loader/user program
because of an undefined reference to a BPF function, however the
object file generated with the clang compiler works as expected.
Here is a simple hello world example:
```d
//hello_world.d
system:
nogc:
extern(C):
(ldc.attributes.section("license")) __gshared immutable(char)[3]
LICENSE = "GPL";
import core.stdc.stdlib;
import core.stdc.stdarg;
import ldc.attributes;
long bpf_trace_printk(const(char)* fmt, uint fmt_size, ...);
(ldc.attributes.section("xdp_md")) struct xdp_md;
(ldc.attributes.section("xdp")) int xdp_prog(xdp_md* ctx)
{
bpf_trace_printk("Hello, eBPF!\n", 14);
return 0;
}
```
Here is the terminal compilation and outputs:
```sh
$ ldc2 --O2 --betterC -c -g --nogc --march=bpf hello_world.d -of
hello_world_d.o
$ llvm-objdump -h hello_world_d.o
hello_world_d.o: file format elf64-bpf
Sections:
Idx Name Size VMA Type
0 00000000 0000000000000000
1 .strtab 000000f6 0000000000000000
2 .text 00000000 0000000000000000 TEXT
3 xdp 00000030 0000000000000000 TEXT
4 .relxdp 00000020 0000000000000000
5 license 00000003 0000000000000000 DATA
6 .rodata.str1.1 0000000e 0000000000000000 DATA
7 .debug_loc 00000023 0000000000000000 DEBUG
8 .debug_abbrev 000000a6 0000000000000000 DEBUG
9 .debug_info 000000d4 0000000000000000 DEBUG
10 .rel.debug_info 00000180 0000000000000000
11 .debug_str 000000f3 0000000000000000 DEBUG
12 .debug_pubnames 00000083 0000000000000000 DEBUG
13 .rel.debug_pubnames 00000010 0000000000000000
14 .debug_pubtypes 0000005d 0000000000000000 DEBUG
15 .rel.debug_pubtypes 00000010 0000000000000000
16 .BTF 00000146 0000000000000000
17 .rel.BTF 00000010 0000000000000000
18 .BTF.ext 00000060 0000000000000000
19 .rel.BTF.ext 00000030 0000000000000000
20 .eh_frame 00000030 0000000000000000 DATA
21 .rel.eh_frame 00000010 0000000000000000
22 .debug_line 00000046 0000000000000000 DEBUG
23 .rel.debug_line 00000010 0000000000000000
24 .symtab 00000120 0000000000000000
$ bpftool btf dump file hello_world_d.o
[1] PTR 'hello_world.xdp_md*' type_id=0
[2] FUNC_PROTO '(anon)' ret_type_id=3 vlen=0
[3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[4] FUNC 'xdp_prog' type_id=2 linkage=global
[5] ARRAY '(anon)' type_id=1 index_type_id=6 nr_elems=3
[6] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32
encoding=(none)
[7] VAR 'LICENSE' type_id=5, linkage=global
[8] DATASEC 'license' size=0 vlen=1
type_id=7 offset=0 size=3 (VAR 'LICENSE')
```
The loading/user program written in D works for kernel programs
written in C, but does not work for D kernel programs and gives
the following output:
```sh
$ ldc2 --betterC -L-lbpf -L-lelf -L-lz -of=loader_d loader_d.d
$ sudo ./loader_d
libbpf: elf: skipping unrecognized data section(20) .eh_frame
libbpf: elf: skipping relo section(21) .rel.eh_frame for
section(20) .eh_frame
libbpf: failed to find BTF for extern 'bpf_trace_printk': -2
ERROR: opening BPF object file failed
```
Attempting to run with the `bpftool`, gives the same response.
The D loading program will be appended at the end for
completeness.
This is the equivalent C kernel program:
```c
//hello_world.c
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
char LICENSE[] SEC("license") = "GPL";
SEC("xdp")
int xdp_prog(struct xdp_md *ctx) {
bpf_trace_printk("Hello, eBPF!\n", 14);
return XDP_PASS;
}
```
Then the compilation and object dump:
```sh
$ clang -O2 -g -target bpf -c hello_world.c -o hello_world.o
$ llvm-objdump -h hello_world.o
hello_world.o: file format elf64-bpf
Sections:
Idx Name Size VMA Type
0 00000000 0000000000000000
1 .strtab 000000fa 0000000000000000
2 .text 00000000 0000000000000000 TEXT
3 xdp 00000030 0000000000000000 TEXT
4 .relxdp 00000010 0000000000000000
5 license 00000004 0000000000000000 DATA
6 .rodata.str1.1 0000000e 0000000000000000 DATA
7 .debug_abbrev 000000f5 0000000000000000 DEBUG
8 .debug_info 00000110 0000000000000000 DEBUG
9 .rel.debug_info 00000040 0000000000000000
10 .debug_str_offsets 00000070 0000000000000000 DEBUG
11 .rel.debug_str_offsets 000001a0 0000000000000000
12 .debug_str 00000132 0000000000000000 DEBUG
13 .debug_addr 00000020 0000000000000000 DEBUG
14 .rel.debug_addr 00000030 0000000000000000
15 .BTF 00000259 0000000000000000
16 .rel.BTF 00000010 0000000000000000
17 .BTF.ext 00000060 0000000000000000
18 .rel.BTF.ext 00000030 0000000000000000
19 .debug_frame 00000028 0000000000000000 DEBUG
20 .rel.debug_frame 00000020 0000000000000000
21 .debug_line 000000a7 0000000000000000 DEBUG
22 .rel.debug_line 00000090 0000000000000000
23 .debug_line_str 00000086 0000000000000000 DEBUG
24 .llvm_addrsig 00000002 0000000000000000
25 .symtab 00000138 0000000000000000
$ bpftool btf dump file hello_world.o
[1] PTR '(anon)' type_id=2
[2] STRUCT 'xdp_md' size=24 vlen=6
'data' type_id=3 bits_offset=0
'data_end' type_id=3 bits_offset=32
'data_meta' type_id=3 bits_offset=64
'ingress_ifindex' type_id=3 bits_offset=96
'rx_queue_index' type_id=3 bits_offset=128
'egress_ifindex' type_id=3 bits_offset=160
[3] TYPEDEF '__u32' type_id=4
[4] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32
encoding=(none)
[5] FUNC_PROTO '(anon)' ret_type_id=6 vlen=1
'ctx' type_id=1
[6] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[7] FUNC 'xdp_prog' type_id=5 linkage=global
[8] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED
[9] ARRAY '(anon)' type_id=8 index_type_id=10 nr_elems=4
[10] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32
encoding=(none)
[11] VAR 'LICENSE' type_id=9, linkage=global
[12] ARRAY '(anon)' type_id=8 index_type_id=10 nr_elems=14
[13] DATASEC 'license' size=0 vlen=1
type_id=11 offset=0 size=4 (VAR 'LICENSE')
```
The C program executes correctly with the loading program. It
does seem to have much more detail than the D object code,
perhaps because things are correctly exported. I tried looking
into the GDC compiler but, did not find a bpf export capability,
and the `--fno-exceptions` flag did not remove the `.eh_frame`
from the obejct file which may be responsible for the warnings in
the D output.
Compilers:
```sh
$ ldc2 --version
LDC - the LLVM D compiler (1.40.0-git-eef3229):
based on DMD v2.110.0 and LLVM 18.1.3
built with LDC - the LLVM D compiler (1.39.0)
Default target: x86_64-pc-linux-gnu
Host CPU: znver4
http://dlang.org - http://wiki.dlang.org/LDC
```
```sh
$ clang --version
Ubuntu clang version 18.1.3 (1ubuntu1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
```
Thanks in advance, sorry that this was a long one.
```d
import core.stdc.stdio;
import core.stdc.stdlib;
extern(C):
struct bpf_object;
struct bpf_program;
struct bpf_link;
struct bpf_object_open_opts;
bpf_object* bpf_object__open_file(const char *path, const
bpf_object_open_opts* opts);
int libbpf_get_error(const bpf_object*);
bpf_program* bpf_object__find_program_by_name(const bpf_object
*obj, const char *name);
int bpf_object__load(bpf_object* obj);
int if_nametoindex(const char*);
int bpf_link__destroy(bpf_link*);
void bpf_object__close(bpf_object*);
bpf_link* bpf_program__attach_xdp(const bpf_program* prog, int
ifindex);
int bpf_link__fd(const bpf_link* link);
void read_trace_pipe(int n)
{
static char[256] buf;
buf[] = '\0';
auto trace_file =
fopen("/sys/kernel/debug/tracing/trace_pipe", "r");
if(trace_file == null)
{
fprintf(stderr, "trace pipe could not be opened!\n");
return;
}
for(int i = 0; i < n; ++i)
{
fprintf(stderr, "print for %d\n", i);
auto nobs = fread(buf.ptr, typeof(buf[0]).sizeof,
buf.length - 1, trace_file);
if(nobs > 0)
{
fprintf(stdout, "%s", buf.ptr);
}
}
fprintf(stdout, "\n");
return;
}
int main()
{
bpf_link* link = null;
bpf_program* prog;
bpf_object* obj;
const char* ifname = "lo";
const char* filename = "hello_world_d.o";
int link_fd;
int ifindex;
obj = bpf_object__open_file(filename, null);
if (libbpf_get_error(obj)) {
fprintf(stderr, "ERROR: opening BPF object file failed\n");
return 0;
}
prog = bpf_object__find_program_by_name(obj, "xdp_prog");
if (!prog) {
fprintf(stderr, "ERROR: finding a prog in obj file failed\n");
goto cleanup;
}
/* load BPF program */
if (bpf_object__load(obj)) {
fprintf(stderr, "ERROR: loading BPF object file failed\n");
goto cleanup;
}
// Get the index of the network interface
ifindex = if_nametoindex(ifname);
if (ifindex == 0) {
fprintf(stderr, "Failed to get interface index");
bpf_object__close(obj);
return 1;
}
// Attach the eBPF program to the network interface using XDP
link = bpf_program__attach_xdp(prog, ifindex);
if (!link) {
fprintf(stderr, "Failed to attach eBPF program");
bpf_object__close(obj);
return 1;
}
// Retrieve the file descriptor for the link
link_fd = bpf_link__fd(link);
if (link_fd < 0) {
fprintf(stderr, "Failed to retrieve BPF link file
descriptor");
bpf_link__destroy(link); // Clean up the link
bpf_object__close(obj);
return 1;
}
read_trace_pipe(5);
goto cleanup;
cleanup:
bpf_link__destroy(link);
bpf_object__close(obj);
return 0;
}
```
Jan 05
On 06/01/2025 3:17 AM, data pulverizer wrote:long bpf_trace_printk(const(char)* fmt, uint fmt_size, ...);libbpf: failed to find BTF for extern 'bpf_trace_printk': -2Compare your D definition against: ```c static long (* const bpf_trace_printk)(const char *fmt, __u32 fmt_size, ...) = (void *) 6; ``` https://github.com/libbpf/libbpf/blob/c5f22aca0f3aa855daa159b2777472b35e721804/src/bpf_helper_defs.h#L185 The reason it cannot find it, is because its looking for a function, when it is a global variable.
Jan 05
On Sunday, 5 January 2025 at 14:27:57 UTC, Richard (Rikki) Andrew Cattermole wrote:On 06/01/2025 3:17 AM, data pulverizer wrote:I have updated the kernel code to: ```d system: nogc: extern(C): (ldc.attributes.section("license")) __gshared immutable(char)[3] LICENSE = "GPL"; import core.stdc.stdlib; import core.stdc.stdarg; import ldc.attributes; // Define a type alias for the function signature alias BpfTracePrintk = long function(const(char)* fmt, uint fmt_size, ...); // Declare and initialize the function pointer __gshared static BpfTracePrintk bpf_trace_printk = cast(BpfTracePrintk) 6; (ldc.attributes.section("xdp_md")) struct xdp_md { uint data; uint data_end; uint data_meta; uint ingress_ifindex; uint rx_queue_index; uint egress_ifindex; } (ldc.attributes.section("xdp")) int xdp_prog(xdp_md* ctx) { bpf_trace_printk("Hello, eBPF!\n", 14); return 0; } ``` And getting a different error: ```sh $ ldc2 --O2 --betterC -c -g --nogc --fno-moduleinfo --march=bpf hello_world.d -of hello_world_d.o $ ldc2 --betterC -L-lbpf -L-lelf -L-lz -of=loader_d loader_d.d $ sudo ./loader_d libbpf: elf: skipping unrecognized data section(21) .eh_frame libbpf: elf: skipping relo section(22) .rel.eh_frame for section(21) .eh_frame libbpf: BTF loading error: -22 libbpf: -- BEGIN BTF LOAD LOG --- magic: 0xeb9f version: 1 flags: 0x0 hdr_len: 24 type_off: 0 type_len: 312 str_off: 312 str_len: 374 btf_total_size: 710 [1] PTR hello_world.xdp_md* type_id=2 Invalid name -- END BTF LOAD LOG -- libbpf: Error loading .BTF into kernel: -22. BTF is optional, ignoring. libbpf: prog 'xdp_prog': BPF program load failed: Invalid argument libbpf: prog 'xdp_prog': -- BEGIN PROG LOAD LOG -- unknown opcode 8d processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 -- END PROG LOAD LOG -- libbpf: prog 'xdp_prog': failed to load: -22 libbpf: failed to load object 'hello_world_d.o' ERROR: loading BPF object file failed ``` I updated `xdp_md` when it complained about it, it looks like it might be having trouble with the type name being prefixed with the module name `hello_world`?long bpf_trace_printk(const(char)* fmt, uint fmt_size, ...);libbpf: failed to find BTF for extern 'bpf_trace_printk': -2Compare your D definition against: ```c static long (* const bpf_trace_printk)(const char *fmt, __u32 fmt_size, ...) = (void *) 6; ``` https://github.com/libbpf/libbpf/blob/c5f22aca0f3aa855daa159b2777472b35e721804/src/bpf_helper_defs.h#L185 The reason it cannot find it, is because its looking for a function, when it is a global variable.
Jan 05
On 06/01/2025 5:27 AM, data pulverizer wrote:
(ldc.attributes.section("xdp_md"))
struct xdp_md {
I don't understand why you gave a type declaration a section.
Its not done on the C side.
https://github.com/libbpf/libbpf/blob/c5f22aca0f3aa855daa159b2777472b35e721804/include/uapi/linux/bpf.h#L6449
And this error would imply it too:
``[1] PTR hello_world.xdp_md* type_id=2 Invalid name``
Jan 05
On Monday, 6 January 2025 at 07:26:02 UTC, Richard (Rikki) Andrew Cattermole wrote:On 06/01/2025 5:27 AM, data pulverizer wrote:I don't actually think it matters what I do with the struct it still gives the same error until I remove the argument completely. So if I just use `struct xdp_md` as in the C code it outputs the same error, and if I use `void *` as the argument, it outputs the same error. When I remove the argument completely I get the error: ```d PTR extern (C) long function(const(char)* fmt, uint fmt_size, ...) nogc system type_id=8 Invalid name ``` I use a cast like this `cast(BpfTracePrintk)6` for the value because D will not allow a cast to `void*` in this case. But all of this may not be that relevant, it looks like from the output, these variables remain as D symbols and are not being translated with the `--march=bpf` switch to instructions the kernel can interpret.(ldc.attributes.section("xdp_md")) struct xdp_md {I don't understand why you gave a type declaration a section. Its not done on the C side. https://github.com/libbpf/libbpf/blob/c5f22aca0f3aa855daa159b2777472b35e721804/include/uapi/linux/bpf.h#L6449 And this error would imply it too: ``[1] PTR hello_world.xdp_md* type_id=2 Invalid name``
Jan 06
"Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> On 07/01/2025 12:03 AM, data pulverizer wrote:On Monday, 6 January 2025 at 07:26:02 UTC, Richard (Rikki) Andrew Cattermole wrote:The only solution I can think of is to use pragma mangle on the struct, but otherwise I think you need to ask on ldc's bug tracker.On 06/01/2025 5:27 AM, data pulverizer wrote:(ldc.attributes.section("xdp_md")) struct xdp_md {I don't understand why you gave a type declaration a section. Its not done on the C side. https://github.com/libbpf/libbpf/blob/ c5f22aca0f3aa855daa159b2777472b35e721804/include/uapi/linux/bpf.h#L6449 And this error would imply it too: ``[1] PTR hello_world.xdp_md* type_id=2 Invalid name``
Jan 06