Efficient FIle IO on Linux

[u/No-Locksmith-7719]

What path to take when parsing large files (on Linux Debian server, the KVM flavour)? Will std::fstream in binary mode do or do we have to look into APIs?

Apart from the performance, the app is memory sensitive, so where to store those read chunks while in a loop? Would a simple char[something] buffer do?

Finally, we would like to avoid those nasty reinterpret_cast<char*> constructs while reading, but are wondering if those are actually used in production? I should also add that the file sizes vary greatly.

[u/not_a_novel_account]

I mean, what's the balance you're trying to strike here between simplicity and performance?

The simple way is fread() into a fixed buffer.

The most performant way is io_uring, for which many high-perf IO frameworks exist to make more usable.

There's nothing nasty about reinterpet_cast<> for dealing with raw binary buffers, it's what the cast exists for. memcpy if you need to start an object lifetime.

[u/UndefFox → u/not_a_novel_account]

From reading online, it seems that io_uring performs better in case of heavy unpredictable and large overrides of the memory, while mmap will still outperform it in plain read-only sequential reads. I've never worked with it, but is it actually guaranteed to be faster?

[u/not_a_novel_account → u/UndefFox]

mmap will stall out on page faults, io_uring's entire original purpose, the reason it exists, was to outperform mmap for local disk IO. It does so.

The trivial explanation is pausing transfer at every 4K boundary is a really bad IO pattern.

[u/UndefFox → u/not_a_novel_account]

Huh, I thought kernel can detect access patterns and will preload pages if we do sequential access only. Thanks for the paper.

[u/not_a_novel_account → u/UndefFox]

The pages are loaded in cache, but they get placed in user space on page faults. Page faults and TLB walks aren't free.

[u/UndefFox]

Wouldn't using mmap be the simplest approach in such a case?

[u/No-Locksmith-7719 → u/UndefFox]

Thanks, certainly an option, we will look into it. We would like to keep the memory consumption at predictable levels and try to avoid dynamic store per process instance/thread if possible. Some of those files can be gigabytes long.

[u/UndefFox → u/No-Locksmith-7719]

As far as I understand, it shouldn't allocate RAM to store files at all, as in directly copying it to the process memory. All chunks are fetched by demand and the main consumption comes from kenrel using file cache to access file faster, so it's basically offloading responsibility of managing the file access onto the kernel.

Tho, I don't think there much in terms of optimization, besides making access pattern predictable, and afaik kernel handles sequential access quite well.

[u/No-Locksmith-7719 → u/UndefFox]

We have some process watcher daemon thingy running in the background. If it sees a services is allocating too much memory, it might panic and kill off our web service, but I am not sure about it 100%. At this point, it's just an assumption.

[u/UndefFox → u/No-Locksmith-7719]

Cached files aren't counted towards the process memory count. When you open any of the performance monitors, even something like htop, it has separate field for cache, so it's handled fully by kernel and takes into count all files cached across all processes, not per process, hence it should not trigger it at all.

[u/aiusepsi]

Regarding reinterpret_cast, if you're reading something like binary integers (or any trivially-copyable type), then a sensible option is using memcpy. This might seem mad, but compilers know about memcpy, and will do the most optimal thing, and it's all well-defined, which is not always the case with reinterpret_cast. And use std::byteswap if you need to convert from big endian to little endian.

I would also second the recommendation of using mmap. It's taking advantage of the kernel's virtual memory system; the file becomes logically part of your process's address space, so you can work on it just like one massive array (which is very convenient), but if any given page is actually resident in RAM is basically up to the kernel. Because it's all managed by the kernel, it can do all sorts of neat tricks. For example, if a file is mmapped read-only into two different processes, the same physical page of RAM can be used by both processes. This sort of thing is why the RAM occupied by memory-mapped files doesn't really count against your process's memory usage.

The kernel can also adapt to the system's overall memory usage. If there's physical RAM to spare, the kernel can keep the entire file resident in RAM. If there isn't any to spare, pages which aren't in use will be dropped from RAM, and reloaded the next time they're accessed, without you having to manage any of this yourself.

[u/No-Locksmith-7719 → u/aiusepsi]

Thanks, I appreciate the info. Indeed that memcpy might take some time adjusting to.

[u/mredding]

Bog standard std::fstream is VERY conservative in it's implementation, and thus will be slow. But it's also very reasonable to use a stream as an interface.

You start with the stream buffer - this class abstracts away the device; in your case, you're abstracting away the platform specific interface you're going to use to access the file.

class big_file: public std::streambuf {
  file_destriptor_type fd;

  int_type underflow() override, overflow(int_type) override;

public:
  big_file(std::filesystem::path);

  rt optimized_code_path(param_type);
};

This is about the minimum you would need. I'm going to leave out a lot of platform specifics, but you could memory map, or you could buffer, or unbuffer - on the file descriptor, or implement your own buffer, which the base class gives you a cursory implementation, or not; you could kernel bypass, implement DMA, IPC, or any other method of moving shit from one point to another.

So then to use it:

big_file bf{get_path()};
std::istream is{&bf};

And you're off to the races. Now you can write ALL your code in terms of streams, and leverage a platform specific high speed interface. The neat thing about streams is that you can streamify anything. You can stream externally, to devices, networks, other processes, or you can stream internally, to other components of your code - widget objects, functions, and the like.

You also don't have to serialize to text. You don't have to actually pass through the stream class or the stream buffer. The crux is you have to change the way you think from imperative and procedural to either OOP or at the very least strong types with semantics. An int is an int, but a weight is not a height, no matter how they're implemented. You need your own types that are aware of their own context within it's own program. In other words:

class weight {
  int value;

  friend std::istream &operator >>(std::istream &is, weight &w) {
    if(auto s = std::istream::sentry{is}; s) {
      if(auto big_buf = dynamic_cast<big_file *>(is.rdbuf()); big_buf) {
        big_buf->optimized_code_path(w);
      }
    } else if(is) {
      is >> w.value;
    }

    return is;
  }
};

That dynamic cast is constant time, this ain't like the compilers as late as of the early 2000s. You can branch predict that cast. You can weight the branch predictor. The cost amortizes to near zero. You have an optimized path for your expensive operations and a universal fallback. You can implement as many stream types, whole hierarchies, and have a tiered approach of the next-most-optimal or configured from available code paths, and this is how and where you do that.

Notice the optimal path doesn't rely on the stream object implementation directly:

weight w;
is >> w;

This reduces to a function call; the stream type resolves it to our implementation at compile-time. The stream object is really only used to get the stream buffer pointer. The sentry class is an obligatory quirk before you touch the stream buffer, if you're going to skip the stream class IO operations and go low level. But then we go to our optimized implementation, which we wrote, so we know exactly what it's going to do in the most optimal way.

Well what of:

std::vector<weight> data;

Here I recommend you make a type:

class weights: std::vector<weight> {
public:
  using std::vector<weight>::vector;
  using std::vector<weight>::emplace_back;

Etcetera. You don't want to load one weight at a time, you want to establish the sentry once and bulk load the data at a time.

Or maybe you make a more optimal type that isn't a vector but a memory map of some block size and uses type punning over memory to access the weights. You can derive the stream type to restrict IO, so your memory mapping types will only work with streams that memory map files. This way your types are safe and their semantics can be enforced at compile-time - you don't have to code EVERYTHING to std::istream.

This is what high performance IO code looks like in C++. Make types around the sorts of things you're doing. You're still using the high performance implementation details you always were, but you implement it within the established framework to grant yourself robustness and flexibility. To define all your own interfaces and semantics would just be reinventing the wheel, and typically it's always rigid and brittle.

[u/TheRavagerSw]

https://github.com/cppfastio/fast_io

[u/xoner2]

Yes. fread into char [fixedSize]. No need to overthink this, solved problem.

[u/KingAggressive1498 → u/xoner2]

if you can fully stream your parser, this works quite well actually, although you can shave a bit by dropping into posix read() instead. use like a 2MB buffer.