www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Creating a pointer/slice to a specific-size buffer? (Handing out a

reply Gavin Ray <ray.gavin97 gmail.com> writes:
Suppose that you have a memory manager, or arena-like class, 
which contains a buffer used to store memory.

And you want to hand out chunks of this memory to other parts of 
your program. These chunks should all be `PAGE_SIZE`.


You might have something like:

```d
enum PAGE_SIZE = 4096;
enum BUF_POOL_NUM_PAGES = 1024;

class BufferPool
{
	align(PAGE_SIZE) ubyte[PAGE_SIZE * BUF_POOL_NUM_PAGES] data;

	this()
	{
		mmap(&data, data.sizeof,
                      PROT_READ | PROT_WRITE, MAP_ANONYMOUS | 
MAP_PRIVATE, -1, 0);
	}
}
```

Now there should be a function, like `get_page()`, that returns a 
`PAGE_SIZE` pointer of `ubyte`. But I can only figure out how to 
return a `ubyte[]`:

```d
ubyte[] get_page(frame_idx_t frame_idx)
{
	return data[frame_idx * PAGE_SIZE .. (frame_idx + 1) * 
PAGE_SIZE];
}
```

I am curious if you can return something like `ubyte[PAGE_SIZE]*` 
or `ref ubyte[PAGE_SIZE]`?

Thank you =)
Jan 13 2023
next sibling parent Gavin Ray <ray.gavin97 gmail.com> writes:
I probably should have mentioned, the equivalent in C++ is the 
below:

```cpp
#include <cstddef>
#include <span>
#include <sys/mman.h>

static constexpr size_t PAGE_SIZE          = 4096;
static constexpr size_t BUF_POOL_NUM_PAGES = 1024;

class BufferPool
{
   private:
     alignas(PAGE_SIZE) std::byte data[BUF_POOL_NUM_PAGES * 
PAGE_SIZE];

   public:
     BufferPool()
     {
         mmap(data, BUF_POOL_NUM_PAGES * PAGE_SIZE,
              PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, 
-1, 0);
     }

     // Return a fat-pointer of PAGE_SIZE bytes over the page, 
containing { T* ptr, len }
     std::span<std::byte, PAGE_SIZE> get_page(size_t page_num)
     {
         return std::span<std::byte, PAGE_SIZE>(data + page_num * 
PAGE_SIZE, PAGE_SIZE);
     }
};
```
Jan 13 2023
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 1/13/23 06:49, Gavin Ray wrote:


 I am curious if you can return something like `ubyte[PAGE_SIZE]*` or
 `ref ubyte[PAGE_SIZE]`?


A simple cast seems to work:

enum PAGE_SIZE = 4096;
enum BUF_POOL_NUM_PAGES = 1024;
alias frame_idx_t = size_t;

ubyte[10_000] data;

ubyte[PAGE_SIZE]* get_page(frame_idx_t frame_idx)
{
     auto ptr = data.ptr + frame_idx * PAGE_SIZE;
     return cast(ubyte[PAGE_SIZE]*)ptr;
}

void main() {
}

Ali
Jan 13 2023
parent reply Gavin Ray <ray.gavin97 gmail.com> writes:
On Friday, 13 January 2023 at 14:57:40 UTC, Ali Çehreli wrote:

 On 1/13/23 06:49, Gavin Ray wrote:


 I am curious if you can return something like


 `ubyte[PAGE_SIZE]*` or

 `ref ubyte[PAGE_SIZE]`?


 A simple cast seems to work:

 enum PAGE_SIZE = 4096;
 enum BUF_POOL_NUM_PAGES = 1024;
 alias frame_idx_t = size_t;

 ubyte[10_000] data;

 ubyte[PAGE_SIZE]* get_page(frame_idx_t frame_idx)
 {
     auto ptr = data.ptr + frame_idx * PAGE_SIZE;
     return cast(ubyte[PAGE_SIZE]*)ptr;
 }

 void main() {
 }

 Ali


Ah, much thanks Ali!

This is "valid" D I hope?

I searched Github for `"ubyte[4096]*"` and I found two uses of 
it, in the `xtrix` repo which seems to be an Operating System 
written in D:

![screenshot-of-ubyte-sized-ptr-on-github](https://i.imgur.com/WaAklbq.png)
Jan 13 2023
next sibling parent reply Gavin Ray <ray.gavin97 gmail.com> writes:
Maybe it would be better to wrap the slice in a new class with an 
invariant?

Because what I want to do is:

1. Ensure that the length of the underlying referenced/pointed-to 
data is `PAGE_SIZE`
2. Benefit from the features of D that I can

The bounds-checking of slices saves a lot of headaches during 
development.

So maybe doing something like this might be better?

```d
struct Frame
{
	ubyte[] data;

	invariant
	{
		assert(data.length == PAGE_SIZE);
	}
}

class BufferPool
{
	align(PAGE_SIZE) ubyte[PAGE_SIZE * BUF_POOL_NUM_PAGES] data;
	Frame[BUF_POOL_NUM_PAGES] frames;

	this()
	{
		// mmap
		foreach (i; 0 .. BUF_POOL_NUM_PAGES)
		{
			frame_idx_t frame_idx = cast(frame_idx_t) i;
			frames[frame_idx].data = data[frame_idx * PAGE_SIZE .. 
(frame_idx + 1) * PAGE_SIZE];
		}
	}
}
```
Jan 13 2023
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 1/13/23 07:22, Gavin Ray wrote:

 Maybe it would be better to wrap the slice in a new class with an
 invariant?


Possibly but please check before using because I think 'invariant' 
requires presence of member functions:

   https://dlang.org/spec/struct.html#Invariant


 Because what I want to do is:

 1. Ensure that the length of the underlying referenced/pointed-to data
 is `PAGE_SIZE`


My first thought was why not use a slice anyway?

Worth noting that static arrays are value types. Also, they all have 
different types from each other and have the potential to cause template 
bloat.


 class BufferPool


Off-topic, most D programmers use struct unless they need class.


      Frame[BUF_POOL_NUM_PAGES] frames;


Makes sense to me.

Ali
Jan 13 2023
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 1/13/23 07:07, Gavin Ray wrote:


 This is "valid" D I hope?


Yes because static arrays are just elements side-by-side in memory. You 
can cast any piece of memory to a static array provided the length and 
alignment are correct.

However, such a cast is not allowed in  safe code.

Ali
Jan 13 2023
parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Jan 13, 2023 at 08:31:17AM -0800, Ali Çehreli via Digitalmars-d-learn
wrote:

 On 1/13/23 07:07, Gavin Ray wrote:
 

 This is "valid" D I hope?


 
 Yes because static arrays are just elements side-by-side in memory.
 You can cast any piece of memory to a static array provided the length
 and alignment are correct.


[...]

Or to be more precise, cast the memory to a *pointer* to a static array
of the right size.  Static arrays are by-value types; passing around the
raw array will cause the array to be copied every time, which is
probably not what is intended.


T

-- 
"You are a very disagreeable person." "NO."
Jan 13 2023
parent Gavin Ray <ray.gavin97 gmail.com> writes:
On Friday, 13 January 2023 at 19:16:17 UTC, H. S. Teoh wrote:

 On Fri, Jan 13, 2023 at 08:31:17AM -0800, Ali Çehreli via 
 Digitalmars-d-learn wrote:

 On 1/13/23 07:07, Gavin Ray wrote:
 

 This is "valid" D I hope?


 
 Yes because static arrays are just elements side-by-side in 
 memory. You can cast any piece of memory to a static array 
 provided the length and alignment are correct.


 [...]

 Or to be more precise, cast the memory to a *pointer* to a 
 static array of the right size.  Static arrays are by-value 
 types; passing around the raw array will cause the array to be 
 copied every time, which is probably not what is intended.


 T


Thanks Teoh and Ali, I wound up passing a pointer to a static 
array.

If anyone is curious, here's the full WIP implementation -- it's 
for a toy database (Postgres-like) I'm writing for a hobby and 
learning project:

https://ldc.godbolt.org/z/Kvh7dv96c
Jan 13 2023