SystemTap is not competition for dtrace. If you've ever been on a system that had dtrace, you'd find that pretty much everything works.
He's going through the same exercise I went through recently where your customer is running your software on Linux and you want to try to make use of the facilities offered by that OS so he can reproduce and diagnose the same issues.
The problem is that Linux just doesn't have good diagnostic tools. Most people don't recognize it because they've never had good tools. It's like being a C programmer who's never heard of a debugger. You can keep adding stuff to your code to try to isolate a bug and keep rerunning tests hoping it hits it where you've instrumented. That seems to be fine. But then you get a debugger and you can suddenly do significantly more on a running system gathering even more information on exactly the part that you care about.
This is what dtrace is. Linux needs something like it. If SystemTap will be that someday, then great. It has to work, though. My attempts to get it functional on Ubuntu and debian left me not feeling too great about the current state of thing (at least on debian, you can get kernel probes in without adding new and poorly documented package repository locations in). In neither case could I get SDTs working, though. For example, with dtrace, I can pretty trivially tell you the exact code paths in any running application on your system that are leading to IOPS. Then maybe I can exclude "known" paths and focus on only areas we don't think there should be any. This will include, for example, paging information. I can decide whether I want that when I'm drilling down. I can then look at another process that's running on the system and see when one is competing with the other for IOPS, CPU time, etc... I do this while the software is running and with minimal impact to the performance of the system.
When SystemTap does these things on my customer's Linux boxes, then I'd call it a dtrace competitor.
If I was your customer, the fact you're trying to get systemtap running on ubuntu rather than RHEL or a derivative would concern me.
Someone should really write an article on using dtrace on a BSD where it's patchable, but not officially supported, and call it 'using dtrace'. The article is very transparent FUD.
Since HN has recently got into the habit of correcting misleading titles, perhaps PG could correct this to 'using SystemTap on Ubuntu'?
Systemtap is made, principally, by Red Hat. On RHEL and similar, running a tap consists of running the following command:
stap tapset.stp
This is fairly well known and easily discoverable.
Ubuntu doesn't even include SystemTap in it's default kernel. Saying that his experiences running SystemTap on Ubuntu are what 'using SystemTap' is like is very misleading.
You don't need SystemTap to determine which process is using I/O. Just use pidstat from the venerable sysstat package that comes with all Linuxes (http://sebastien.godard.pagesperso-orange.fr/man_pidstat.htm...). Requires Linux 2.6.20 or later to report I/O usage. You can also just look in /proc, which has had I/O counters a few years now.
Comments
On dtrace.org, huh? I didn't realize dtrace as a project has fall so low as to poo-poo the competition.
SystemTap is not competition for dtrace. If you've ever been on a system that had dtrace, you'd find that pretty much everything works.
He's going through the same exercise I went through recently where your customer is running your software on Linux and you want to try to make use of the facilities offered by that OS so he can reproduce and diagnose the same issues.
The problem is that Linux just doesn't have good diagnostic tools. Most people don't recognize it because they've never had good tools. It's like being a C programmer who's never heard of a debugger. You can keep adding stuff to your code to try to isolate a bug and keep rerunning tests hoping it hits it where you've instrumented. That seems to be fine. But then you get a debugger and you can suddenly do significantly more on a running system gathering even more information on exactly the part that you care about.
This is what dtrace is. Linux needs something like it. If SystemTap will be that someday, then great. It has to work, though. My attempts to get it functional on Ubuntu and debian left me not feeling too great about the current state of thing (at least on debian, you can get kernel probes in without adding new and poorly documented package repository locations in). In neither case could I get SDTs working, though. For example, with dtrace, I can pretty trivially tell you the exact code paths in any running application on your system that are leading to IOPS. Then maybe I can exclude "known" paths and focus on only areas we don't think there should be any. This will include, for example, paging information. I can decide whether I want that when I'm drilling down. I can then look at another process that's running on the system and see when one is competing with the other for IOPS, CPU time, etc... I do this while the software is running and with minimal impact to the performance of the system.
When SystemTap does these things on my customer's Linux boxes, then I'd call it a dtrace competitor.
If I was your customer, the fact you're trying to get systemtap running on ubuntu rather than RHEL or a derivative would concern me.
Someone should really write an article on using dtrace on a BSD where it's patchable, but not officially supported, and call it 'using dtrace'. The article is very transparent FUD.
dtrace.org is not affiliated with dtrace-the-project. It's just a bunch of ex-Sun guys who share the domain.
Since HN has recently got into the habit of correcting misleading titles, perhaps PG could correct this to 'using SystemTap on Ubuntu'?
Systemtap is made, principally, by Red Hat. On RHEL and similar, running a tap consists of running the following command:
This is fairly well known and easily discoverable.Ubuntu doesn't even include SystemTap in it's default kernel. Saying that his experiences running SystemTap on Ubuntu are what 'using SystemTap' is like is very misleading.
You don't need SystemTap to determine which process is using I/O. Just use pidstat from the venerable sysstat package that comes with all Linuxes (http://sebastien.godard.pagesperso-orange.fr/man_pidstat.htm...). Requires Linux 2.6.20 or later to report I/O usage. You can also just look in /proc, which has had I/O counters a few years now.