Can someone explain what “the framebuffer” is? I’m familiar with OpenGL programming where the OS can provide a framebuffer for an application but I’m confused about whether there is a global framebuffer for the entire desktop. Is this a Linux specific concept?
As far as I know, a framebuffer can mean a lot of things depending on hardware and implementation, but it was used to refer to actual memory that would contain pixel values that would eventually be written to the screen. In Linux, this is abstracted by the framebuffer device, which is hardware independent (you can actually have several fbdevices, which if I'm not mistaken end up referring to different monitors usually). What's convenient about the implementation is that these devices still work as normal memory devices, which means you can read/write as you would any other memory. Some more info: https://www.kernel.org/doc/html/latest/fb/framebuffer.html
I'll preface this by saying that I may have some misconceptions. Other people much more knowledgeable than I am have posted summaries of how modern graphics hardware works on HN before.
My understanding is that modern hardware is significantly more complicated at the lowest levels and (at least generally) no longer has a dedicated framebuffer (at least in the same sense that old hardware did).
My understanding of the memory access provided by fbdev is that it's an extremely simple API. In other words an outdated abstraction that's still useful so it's kept around.
An example of this complexity is video streams utilizing hardware accelerated decoding. Those often won't show up in screenshots, or if they do they might be out of sync or otherwise not quite what you saw on screen because the driver is attempting to construct a single cohesive snapshot for you where one never actually existed in the first place.
If I got anything wrong please let me know. My familiarity generally stops at the level of the various cross platform APIs (opengl, vulkan, opencl, etc).
My understanding is that modern hardware is significantly more complicated at the lowest levels and (at least generally) no longer has a dedicated framebuffer (at least in the same sense that old hardware did).
Modern hardware still generally can be put into a default VGA-compatible[1] mode that does use a dedicated framebuffer. This mode is used by the BIOS and during boot until the model-specific GPU driver takes over.
My understanding of the memory access provided by fbdev is that it's an extremely simple API.
Maybe some of fbdev are like that, but most of them are not. They use vga/vesa interfaces to get a real video memory and write into it. A text console is also using vga video memory to write character data into it.
I still wonder do there any ways to use VGA at its full. Like loading sprites into invisible on the screen video memory and copying them into their right place on the screen. VGA allowed to copy 8 of 4-bit pixels by copying one byte, for example. Were these things just dropped off for a nice abstraction, or maybe there is some ioctls to switch modes for read/writes into video memory? I don't know and never was interested enough to do a research.
In other words an outdated abstraction that's still useful so it's kept around.
Yes, it is kinda like this, but the outdated abstraction is realized on video card, kernel just gives access to it.
In Linux fbdev is more like a fallback device when drivers for a specific video card are not accessible. fbdevs are used to make a text console with more than 80x25 characters. Video acceleration or opengl can work on fbdev only as a software implementation.
Most GPU drivers these days are DRM drivers, which implement fbdev support for backwards compatibility only [0]. The fbdev API is primarily "fake" these days.
DRM/KMS using dumb buffers are the preferred API if you want to do software rendering and modesetting. You can find several examples of this online if you search for drm_mode_create_dumb.
Unless you're deep in a conversation with a graphics nerd, when "the framebuffer" is referenced, what the person normally means is some area of memory, accessible programmatically, that directly represents the pixels displayed on the screen. No fancy windows, vectors, coordinates, just raw memory and values that are the literal values the screen is showing.
In practice, it's not literally that, but in practice, it acts/works like that.
Most generally, it's a place you can buffer a frame. Old-school computers usually have an area of memory set aside and they're continually reading this area and sending its contents to the TV screen, so if you set bytes in this area they instantly appear as pixels on the TV.
Even with modern graphics cards there's a global framebuffer for the entire desktop. It's updated every frame by the operating system sending commands to the graphics card to copy data from all over the place from different programs.
Linux's fbdev API (/dev/fb0) provides a framebuffer that looks like a file - you can read, write and mmap it. This is deprecated, not because it's obsolete functionality, but because the API is obsolete as they want to replace it with DRM dumb buffers (that's Direct Rendering Manager, not the evil thing). You have to use the whole DRM system, which lets you select which graphics card to use, which video ports to use if your card has more than one, which resolution, etc. You can then allocate a frame buffer and tell it to display that buffer. You can also allocate more than one buffer and tell the card when to switch to a different one (i.e. double-buffering).
Plane independent page flips - No need to be tied to global compositor page-flip present rate, reduced latency, independent timing.
There's at least a buffer for scanout but that doesn't necessarily have to be larger than one line and I don't believe it's end use accessible.
As an example. Consider running a game at one refresh rate in windowed fullscreen (or whatever it's called) mode with your compositor at a different refresh rate. With your desktop spanning two monitors, one operating at 60 Hz the other at 50 Hz. And throw in a media player with a window that's split across those two monitors rendering a 29.97 fps (ie NTSC) video stream via hardware accelerated decoding.
On Linux and on other operating systems that have reused the Linux DRM drivers, you can run OpenGL applications from a virtual terminal text console. Examples are kmscube [1] and the glmark2 benchmark suite.
Comments
Can someone explain what “the framebuffer” is? I’m familiar with OpenGL programming where the OS can provide a framebuffer for an application but I’m confused about whether there is a global framebuffer for the entire desktop. Is this a Linux specific concept?
As far as I know, a framebuffer can mean a lot of things depending on hardware and implementation, but it was used to refer to actual memory that would contain pixel values that would eventually be written to the screen. In Linux, this is abstracted by the framebuffer device, which is hardware independent (you can actually have several fbdevices, which if I'm not mistaken end up referring to different monitors usually). What's convenient about the implementation is that these devices still work as normal memory devices, which means you can read/write as you would any other memory. Some more info: https://www.kernel.org/doc/html/latest/fb/framebuffer.html
I'll preface this by saying that I may have some misconceptions. Other people much more knowledgeable than I am have posted summaries of how modern graphics hardware works on HN before.
My understanding is that modern hardware is significantly more complicated at the lowest levels and (at least generally) no longer has a dedicated framebuffer (at least in the same sense that old hardware did).
My understanding of the memory access provided by fbdev is that it's an extremely simple API. In other words an outdated abstraction that's still useful so it's kept around.
An example of this complexity is video streams utilizing hardware accelerated decoding. Those often won't show up in screenshots, or if they do they might be out of sync or otherwise not quite what you saw on screen because the driver is attempting to construct a single cohesive snapshot for you where one never actually existed in the first place.
If I got anything wrong please let me know. My familiarity generally stops at the level of the various cross platform APIs (opengl, vulkan, opencl, etc).
Modern hardware still generally can be put into a default VGA-compatible[1] mode that does use a dedicated framebuffer. This mode is used by the BIOS and during boot until the model-specific GPU driver takes over.
[1]: https://en.wikipedia.org/wiki/Video_Graphics_Array#Use
Maybe some of fbdev are like that, but most of them are not. They use vga/vesa interfaces to get a real video memory and write into it. A text console is also using vga video memory to write character data into it.
I still wonder do there any ways to use VGA at its full. Like loading sprites into invisible on the screen video memory and copying them into their right place on the screen. VGA allowed to copy 8 of 4-bit pixels by copying one byte, for example. Were these things just dropped off for a nice abstraction, or maybe there is some ioctls to switch modes for read/writes into video memory? I don't know and never was interested enough to do a research.
Yes, it is kinda like this, but the outdated abstraction is realized on video card, kernel just gives access to it.
In Linux fbdev is more like a fallback device when drivers for a specific video card are not accessible. fbdevs are used to make a text console with more than 80x25 characters. Video acceleration or opengl can work on fbdev only as a software implementation.
Most GPU drivers these days are DRM drivers, which implement fbdev support for backwards compatibility only [0]. The fbdev API is primarily "fake" these days.
DRM/KMS using dumb buffers are the preferred API if you want to do software rendering and modesetting. You can find several examples of this online if you search for drm_mode_create_dumb.
[0] https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds...
the frame
the buffer
Other way around
The buffer is between your pixels and the actual analog or digital output. It serves the conversion between the two formats.
Unless you're deep in a conversation with a graphics nerd, when "the framebuffer" is referenced, what the person normally means is some area of memory, accessible programmatically, that directly represents the pixels displayed on the screen. No fancy windows, vectors, coordinates, just raw memory and values that are the literal values the screen is showing.
In practice, it's not literally that, but in practice, it acts/works like that.
Most generally, it's a place you can buffer a frame. Old-school computers usually have an area of memory set aside and they're continually reading this area and sending its contents to the TV screen, so if you set bytes in this area they instantly appear as pixels on the TV.
Even with modern graphics cards there's a global framebuffer for the entire desktop. It's updated every frame by the operating system sending commands to the graphics card to copy data from all over the place from different programs.
Linux's fbdev API (/dev/fb0) provides a framebuffer that looks like a file - you can read, write and mmap it. This is deprecated, not because it's obsolete functionality, but because the API is obsolete as they want to replace it with DRM dumb buffers (that's Direct Rendering Manager, not the evil thing). You have to use the whole DRM system, which lets you select which graphics card to use, which video ports to use if your card has more than one, which resolution, etc. You can then allocate a frame buffer and tell it to display that buffer. You can also allocate more than one buffer and tell the card when to switch to a different one (i.e. double-buffering).
Is there though?
https://docs.kernel.org/gpu/amdgpu/display/mpo-overview.html
There's at least a buffer for scanout but that doesn't necessarily have to be larger than one line and I don't believe it's end use accessible.
As an example. Consider running a game at one refresh rate in windowed fullscreen (or whatever it's called) mode with your compositor at a different refresh rate. With your desktop spanning two monitors, one operating at 60 Hz the other at 50 Hz. And throw in a media player with a window that's split across those two monitors rendering a 29.97 fps (ie NTSC) video stream via hardware accelerated decoding.
On Linux and on other operating systems that have reused the Linux DRM drivers, you can run OpenGL applications from a virtual terminal text console. Examples are kmscube [1] and the glmark2 benchmark suite.
[1] https://gitlab.freedesktop.org/mesa/kmscube
See some details here: https://lwn.net/Articles/955708/