|
|
|
01-14-2009, 12:42 PM
|
#1
|
Human being with feelings
Join Date: Dec 2007
Location: east coast of Kansas
Posts: 681
|
Using dpclat and RATT to diagnose audio performance problems
This is intended to be a friendly guide to using these two fine (and free!) tools to diagnose (not fix) problems with clicking and popping audio.
dpclat is a tool for visualizing the latency of deferred procedure calls by kernel-mode device drivers. In doing so, it also makes a good assessment tool for your system's ability to handle streams of audio or video. It's extremely well described on the linked page, so please do read that page and familiarize yourself with the background.
To use dpclat, you just run the executable (there is no installation needed). You will get a window with an updating bar graph showing you the time taken for deferred procedure calls (DPC) to all the drivers in the system. What it does not show is which driver is responsible if you have a problem.
The tool will give you a sort of "go/no go" visualization. If you get all green bars, your system (hardware and installed drivers) is generally well-equipped for streaming. If you get red bars, the tool is alerting you to the presence of a problem, but that's about it--finding which driver is responsible will be done with another tool altogether.
RATTV3 is that other tool. As described on the linked page, "RATTV3 is designed to help developers of drivers and other kernel mode components audit the ISR and DPC execution time of their components."
However, we will use it a little differently. In cases where we have red bars showing in dpclat window, we'll use RATT to find out where to start looking for the problem driver(s). What we do once we ind them is out of scope for this tutorial--I'm just showing you how to use the tool.
RATT gets installed and then runs as a system tray icon. It will start every time you boot until uninstalled, so make sure you are paying attention and don't leave it running after you are done troubleshooting.
With RATT running, take whatever steps are needed to reproduce your audio issue. You'll want to make sure to leave RATT on for at least three minutes to ensure you get a human-readable log file built. RATT will build a log file in the directory %system32%\logfiles\rattv3 based on the info it collects about DPC times. (note: %system32% is an environment variable--it can point to a different physical path depending on how you chose to install Windows. Look for a folder system32 under the folder where windows is installed.)
You then will open up the above directory in explorer, and open a text file named <machine-name>.cswa-accumulator-report.txt. Wordpad works much better than notepad for reading this file.
Inside this file you'll find a section for each kernel mode driver (DRIVERNAME.SYS). Each section is a histogram (sort of) of DPC times. There are 1109 possible "buckets" a driver call can fall into. A bucket is a range of duration times. What we are looking for is when calls to a given driver take longer than a few hundred microseconds. when we find that, we have identified a driver that could be behaving better.
The easiest way to spot the misbehavior is to look at the first column. the buckets we don't want to see will be numbered above 100. Ignore anything with a bucket number under 50. Those are too fast to matter. Any driver with items in buckets numbered over 109 is doing DPCs that take longer than a millisecond (1000 microseconds), and may be contributing to our performance issues. Further troubleshooting will be needed to determine that.
So a "healthy" driver would have a histogram like this:
Code:
Cumulative Histogram for: Ntfs.sys (Ntfs.sys)
#, Range, Label, ISR Count, DPC Count, DPCTmr Count
10, 10.00us to 19.99us, <20.00us, 0, 0, 63
11, 20.00us to 29.99us, <30.00us, 0, 0, 11
TOTALS, 0, 0, 74
And a driver that deserves further scrutiny would appear like this:
Code:
Cumulative Histogram for: atapi.sys (atapi.sys)
#, Range, Label, ISR Count, DPC Count, DPCTmr Count
11, 20.00us to 29.99us, <30.00us, 0, 2269, 0
12, 30.00us to 39.99us, <40.00us, 0, 4576, 0
13, 40.00us to 49.99us, <50.00us, 0, 446, 0
14, 50.00us to 59.99us, <60.00us, 0, 70, 0
15, 60.00us to 69.99us, <70.00us, 0, 82, 0
16, 70.00us to 79.99us, <80.00us, 0, 49, 0
17, 80.00us to 89.99us, <90.00us, 0, 24, 0
18, 90.00us to 99.99us, <100.00us, 0, 41, 0
19, 100.00us to 109.99us, <110.00us, 0, 136, 0
[more like this...]
302, 2.93ms to 2.94ms, <2.94ms, 1, 0, 0
342, 3.33ms to 3.34ms, <3.34ms, 1, 0, 0
351, 3.42ms to 3.43ms, <3.43ms, 1, 0, 0
415, 4.06ms to 4.07ms, <4.07ms, 1, 0, 0
TOTALS, 7841, 7841, 0
So, low bucket numbers: good, high bucket numbers: bad.
There are also three different times being analyzed by RATT: execution times of interrupt service routines (ISR), deferred procedure calls (DPC), and timer DPCs (DPCTmr). They are different, but effectively all are covered by the low bucket numbers: good, high bucket numbers: bad mantra above.
What next, once you've drawn a bead on a specific driver?
+ Selectively disable suspect devices with device manager to see if your issues get cleared up.
+ get updated drivers for the component if possible.
+ check the hardware behind the driver for performance issues.
It's worth saying--long times aren't a smoking gun here, they are a clue to the real issue. They should be used to narrow down the search field for your cause of your performance issue--it's still up to you to determine the resolution..
Hopefully this is of use to some of you.
|
|
|
01-14-2009, 03:00 PM
|
#2
|
Super Moderator (no feelings)
Join Date: Dec 2007
Location: On or near a dike
Posts: 9,836
|
Good job! I bookmarked this in my "Reaper resources" folder and linked to it for some poor guys who have DPC troubles with their new Lenovo Thinkpad SL laptops.
|
|
|
01-14-2009, 04:44 PM
|
#3
|
Human being with feelings
Join Date: Mar 2008
Location: Sydney, Australia
Posts: 3,955
|
great post. can this be made sticky?
|
|
|
01-16-2009, 05:08 PM
|
#4
|
Human being with feelings
Join Date: Jul 2006
Location: stuck in transition
Posts: 1,870
|
yikes! I ran ratt and I have several drivers above 109. but i'm stuck now cuz I don't know how to disable any of them and i'm tired of googling for an answer. ???
HDAudBus.sys
portcls.sys
rdbss.sys
sthda.sys
tcpip.sys
TDI.SYS
USBPORT.SYS
ntkrnlpa.exe
ACPI.sys
atapi.sys
NDIS.sys
Last edited by politcat; 01-16-2009 at 05:25 PM.
|
|
|
01-16-2009, 10:44 PM
|
#5
|
Human being with feelings
Join Date: Dec 2007
Location: east coast of Kansas
Posts: 681
|
Out of your list, the thing that kind of jumps out at me is the first one. That's the stock driver for the Intel chipset onboard audio for your motherboard.
If you are using the onboard audio chip, it may not be feasible (as a solution) to disable it, but I'd try deactivating your onboard sound in the BIOS setup screen (as a diagnostic).
__________________
"Well feeling (emotion) combined with an artist's discipline is the rarest thing in the world."
-- Ursula Nordstrom
|
|
|
01-17-2009, 10:10 AM
|
#6
|
Human being with feelings
Join Date: Jul 2006
Location: stuck in transition
Posts: 1,870
|
I didn't see anywhere in the bios setup screen that the onboard sound can be disabled
|
|
|
02-03-2009, 04:47 AM
|
#7
|
Human being with feelings
Join Date: Jun 2008
Posts: 13
|
So,
Does anyone have any thoughts on tracking down a driver that's listed as Unknown which is causing the issue? Here's the details:
Cumulative Histogram for: Unknown (Unknown)
# Range Label ISR Count DPC Count DPCTmr Count
0 0.00us to 0.99us <1.00us 0 1882 0
1 1.00us to 1.99us <2.00us 7015368 37918 2
2 2.00us to 2.99us <3.00us 32596420 164608 434
3 3.00us to 3.99us <4.00us 1064039 157747 50773
4 4.00us to 4.99us <5.00us 46677 72477 108547
5 5.00us to 5.99us <6.00us 137372 133734 32974
6 6.00us to 6.99us <7.00us 134460 64185 98156
7 7.00us to 7.99us <8.00us 226479 43175 40970
8 8.00us to 8.99us <9.00us 147192 16215 51761
9 9.00us to 9.99us <10.00us 80008 10062 17049
10 10.00us to 19.99us <20.00us 527600 51398 134029
11 20.00us to 29.99us <30.00us 375807 234711 105331
12 30.00us to 39.99us <40.00us 14491 104751 12448
13 40.00us to 49.99us <50.00us 212 61401 203
14 50.00us to 59.99us <60.00us 1 110689 9
15 60.00us to 69.99us <70.00us 0 128944 0
16 70.00us to 79.99us <80.00us 0 10250 0
17 80.00us to 89.99us <90.00us 0 2157 0
18 90.00us to 99.99us <100.00us 0 631 0
19 100.00us to 109.99us <110.00us 0 337 0
20 110.00us to 119.99us <120.00us 0 117 0
21 120.00us to 129.99us <130.00us 14 13 0
22 130.00us to 139.99us <140.00us 8 7 0
23 140.00us to 149.99us <150.00us 2 9 0
24 150.00us to 159.99us <160.00us 0 1 0
25 160.00us to 169.99us <170.00us 0 2 0
26 170.00us to 179.99us <180.00us 0 1 0
28 190.00us to 199.99us <200.00us 0 1 0
29 200.00us to 209.99us <210.00us 0 1 0
37 280.00us to 289.99us <290.00us 0 1 0
1011 30.00ms to 39.99ms <40.00ms 0 1 0
1012 40.00ms to 49.99ms <50.00ms 0 10 0
1013 50.00ms to 59.99ms <60.00ms 0 2 0
1019 110.00ms to 119.99ms <120.00ms 1 0 0
1020 120.00ms to 129.99ms <130.00ms 1 0 0
1021 130.00ms to 139.99ms <140.00ms 0 8 0
1022 140.00ms to 149.99ms <150.00ms 0 2 0
1027 190.00ms to 199.99ms <200.00ms 0 2 0
TOTALS 42366152 1407450 652686
According to the Rattv3 docs, the "Unknown" thing is a known bug, but I doubt it will ever be fixed.
Thanks,
Jeff
|
|
|
03-17-2009, 10:10 PM
|
#8
|
Mortal
Join Date: Jan 2006
Location: Wickenburg, Arizona
Posts: 14,051
|
Please come to chat sometime
|
|
|
03-18-2009, 04:35 PM
|
#9
|
Mortal
Join Date: Jan 2006
Location: Wickenburg, Arizona
Posts: 14,051
|
can someone help me with mine?
https://stash.reaper.fm/oldsb/131691/ratt-report.zip
so far it looks like
ipnat.sys
videoprt.sys
portcls.sys
sthda.sys (no surprise there)
tcpip.sys
usbprt.sys (youch, I often use a usb soundcard!)
ntkrnlpa.sys
acpi.sys
atapi.sys
iaStor.sys
NDIS.sys
at the very least are the ones messing with me...what can I do?
|
|
|
03-18-2009, 07:07 PM
|
#10
|
Human being with feelings
Join Date: Dec 2007
Location: east coast of Kansas
Posts: 681
|
I'd go into device manager and disable your SigmaTel onboard audio. That latency is ridiculous. And you seem to be using a MOTU card for the "Real" audio. It may be necessary to hit the BIOS to turn it all the way off.
I'd also take a look at disabling the nic. The NDIS.sys numbers are insane.
Do you have any USB stuff attached? I would look for any high-resolution mice or things like that and pull them if the above don't help, but I think those two are the likeliest sources of problems.
__________________
"Well feeling (emotion) combined with an artist's discipline is the rarest thing in the world."
-- Ursula Nordstrom
|
|
|
03-19-2009, 03:16 PM
|
#11
|
Mortal
Join Date: Jan 2006
Location: Wickenburg, Arizona
Posts: 14,051
|
whats nic and ndis?
I use the HDA audio when I am not using real soundcards...how did you know about the motu? Are motu drivers running even though the motu interface is 90 miles away? the run had no usb devices plugged in while rat was monitoring
|
|
|
05-30-2010, 06:50 AM
|
#12
|
Human being with feelings
Join Date: May 2010
Location: Staffordshire, England, UK
Posts: 1
|
DPCs, ISRs, Latency, DPCLat and RATTv3
Guys,
This has proven useful:
http://www.resplendence.com/latmon
in tracking down problematic services, et al. You may want to take a look!
I've found it more useful (and more 'accurate'!?) than "DPCLat.exe" by TheSyCon in identifying dodgy routines/drivers and less-intensive (in terms of its 'learning curve') than "RATT".
There are 3 Tabs: a "Latency" overview, an "Activity" tab (the useful one, with all the ISR/DPC detail) and one with some perfunctory info about "Drivers".
From a purely personal perspective, I have stopped all networking hardware and all related services, but have a major problem with TCPIP.sys interrupting my music with glitches through copious DPCs (as well as moderate - but presumably necessary - nonsense from ACPI.sys too) on my Acer 7720G (3GB, Vista, fully-patched).
Anyone have any ideas on why TCPIP.sys is such a problem and/or whether I can disable it entirely (probably through SysInternals' "AutoRuns" app)?
I dual-boot this Vista machine - and am lucky enough to actually have a couple of physical Acronis-managed boot drives in it (rather than multiple Windows partitions on a single physical drive) - but, if necessary, I don't mind having no Internet access on the 'Music DAW' drive, if it will cure the nasty and unpredictable audio behaviour on such a well-spec'd laptop.
Whilst I'd like to have the ability for network access from this laptop, when booted as a Music DAW - for s/w updates, maintenance, et al - I'll readily forsake it in order to make some half-decent music!!!
So, what to do ...?
All reasonable help welcomed.
Cheers,
Steve.
|
|
|
06-18-2010, 04:58 AM
|
#13
|
Human being with feelings
Join Date: May 2010
Posts: 17
|
I'm running Windows Vista Premium on a Compaq Presario CQ60-224 NR laptop with 4 gigs o' ram, dual Intel T3400 2.16/2.17 mHz, onboard Intel 4500 graphics chip, and a Tascam US-144mkii USB interface.
After a factory install with up-to-date drivers, Reaper and GTR3 sound horrible. After making major changes to the settings as described in this and other threads, I've gotten a HUGE performance boost. I went from lots of red spikes in dpcl to an occassional spike, mostly green with some yellow. Currently, I'm running at anywhere between 51us, mosty around 1000, with hardly anything over 2000, with an occasional spike...one hit 3006us during the writing of this post. It looks like one driver is real bad, the rest...
I installed RATT. Here's the results:
--------------------------------------------------------------------------------
| Counts by driver for: Laptop
| From: FRI JUN-18-2010 11:18:6.69
| To: FRI JUN-18-2010 11:36:1.16
| Runs: 5
--------------------------------------------------------------------------------
Cumulative Histogram for: ()
#, Range, Label, ISR Count, DPC Count, DPCTmr Count
1, 1.00us to 1.99us, <2.00us, 40, 0, 0
2, 2.00us to 2.99us, <3.00us, 5227, 0, 0
3, 3.00us to 3.99us, <4.00us, 438650, 0, 0
4, 4.00us to 4.99us, <5.00us, 1140147, 0, 0
5, 5.00us to 5.99us, <6.00us, 909959, 0, 0
6, 6.00us to 6.99us, <7.00us, 674598, 0, 0
7, 7.00us to 7.99us, <8.00us, 253542, 0, 0
8, 8.00us to 8.99us, <9.00us, 41236, 0, 0
9, 9.00us to 9.99us, <10.00us, 22701, 0, 0
10, 10.00us to 19.99us, <20.00us, 61242, 0, 0
11, 20.00us to 29.99us, <30.00us, 5593, 0, 0
12, 30.00us to 39.99us, <40.00us, 635, 0, 0
13, 40.00us to 49.99us, <50.00us, 185, 0, 0
14, 50.00us to 59.99us, <60.00us, 31, 0, 0
15, 60.00us to 69.99us, <70.00us, 1, 0, 0
21, 120.00us to 129.99us, <130.00us, 173, 0, 0
22, 130.00us to 139.99us, <140.00us, 2519, 0, 0
23, 140.00us to 149.99us, <150.00us, 3115, 0, 0
24, 150.00us to 159.99us, <160.00us, 1072, 0, 0
25, 160.00us to 169.99us, <170.00us, 1207, 0, 0
26, 170.00us to 179.99us, <180.00us, 1132, 0, 0
27, 180.00us to 189.99us, <190.00us, 907, 0, 0
28, 190.00us to 199.99us, <200.00us, 551, 0, 0
29, 200.00us to 209.99us, <210.00us, 187, 0, 0
30, 210.00us to 219.99us, <220.00us, 44, 0, 0
31, 220.00us to 229.99us, <230.00us, 28, 0, 0
32, 230.00us to 239.99us, <240.00us, 14, 0, 0
33, 240.00us to 249.99us, <250.00us, 6, 0, 0
34, 250.00us to 259.99us, <260.00us, 13, 0, 0
35, 260.00us to 269.99us, <270.00us, 5, 0, 0
36, 270.00us to 279.99us, <280.00us, 1, 0, 0
37, 280.00us to 289.99us, <290.00us, 3, 0, 0
38, 290.00us to 299.99us, <300.00us, 2, 0, 0
39, 300.00us to 309.99us, <310.00us, 1, 0, 0
40, 310.00us to 319.99us, <320.00us, 1, 0, 0
42, 330.00us to 339.99us, <340.00us, 3, 0, 0
44, 350.00us to 359.99us, <360.00us, 1, 0, 0
48, 390.00us to 399.99us, <400.00us, 1, 0, 0
TOTALS, 3564773, 0, 0
I've disabled the nic, cam, USB card reader, onboard fax, and the onboard sound. This helped, but not much.
What really improved performance was turning off the page file and a lower screen resolution. After doing these two things, it flies...as compared to the factory install.
Does the report show where the problem is (it's all gibberish to me)?
Thanks!
|
|
|
Thread Tools |
|
Display Modes |
Hybrid Mode
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT -7. The time now is 04:02 AM.
|