Netmon vs Chimney

I recently encountered TCP Chimney for the first time in the wild.


Short version: Chimney is an offload technology that allows the NIC to deal with up to X TCP connections, with any overflow being handled by Windows. All good: get the NIC dealing with more networky stuff, and reduce CPU use. Excellent!


The reason it came up:


I was staring at a small network monitor capture (should have been much bigger) that should have had a few tens of megabytes of FTP but was mostly comprised of SYN, SYN-ACK, ACK s to port 21.


A lot! It’d look like


SYN -> 21, source port X
SYN-ACK
ACK
SYN -> 21, source port X+1
SYN-ACK
ACK


…Hundreds and hundreds of TCP 3-way handshakes, but next to no actual data sent. The server didn’t even appear to be sending its connection banner!


Very, very rarely, I’d actually see a frame or two of FTP traffic, but I thought the symptom I was looking at was indicative of resource starvation on the FTP server.


Perfmon didn’t confirm the diagnosis, and the FTP server logs showed it was transferring loads and loads of data; I just couldn’t see it in the capture.


After ruling out network adapter teaming (The Old Enemy), I wondered if something from the Scalable Networking Pack might have been involved, and a quick internal search later, whop! A symptom match! Because the NIC handles the heavy lifting of all TCP work with Chimney enabled, after the TCP session is established, Netmon doesn’t get to see the traffic!


To disable Chimney so you’re able to gather captures for troubleshooting purposes, you can use the following netsh command:



Netsh int ip set chimney DISABLED


Once that’s done, Netmon (and presumably other NDIS capture drivers, like WinPCap (ethereal/wireshark) should be able to capture all traffic, not just non-TCP stuff!

Fear The Hand Of Non-Selection

One of those “remember it for when you need it” posts:

On Tiny, I’ve been unable to select text in the Outlook preview pane for about a month. When I tried to select things, I’d get a not-quite-looking-like-the-web-hand hand instead of a selection bar, and it would grab at the document rather than just sorta sitting there browser style.

It hasn’t been a big deal, but I couldn’t find anywhere to turn it off in the options, or work out what was happening.

I wondered if it might have been new-fangled Outlook behaviour on Tablet PCs, but couldn’t reproduce it on any others.

Cutting a long story short: I found the culprit – it kinda was a tablet thing, but more due to my use of the scroll bar on a tablet.

At the top of Word scroll bars (and I guess this flows forward into Outlook 2007, which uses Word for email rendering), just above the top scroll button, there’s a little hand.image

Clicking that little hand (intentionally) puts you into drag-the-document-but-don’t-modify-or-select-anything mode.

Clicking that little hand (unintentionally, and less likely to happen if you have a mouse wheel rather than a tablet) will cause the same effect, only you won’t be able to work out what happened or how to undo it! 🙂

I happily accidentally ended up in scrolly mode today while reviewing a Word document, but noticed the effect and was still staring at the button I’d failed to tap.

Yay!

Generating a memory dump when a particular event log occurs

Building from Paul Long’s post on stopping a Netmon3 capture on a particular event, we’re going to re-jig it so that we can run ADPLUS soon after a particular event is logged.

This won’t produce a dump at the exact moment of the event log (which more often is what would be most useful for debugging purposes), but for the case I’m working on, I’m mainly interested in the state of the process at around or after that time; it’s an intermittent problem that we’re not confident we can catch quickly enough manually.

An alternative approach would be to attach, set a break on the Windows event logging function and evaluate the arguments passed to it; that’s not what I’m doing here, and this has the advantage of being able to survive process recycling without special care.

EvtMon.vbs:

‘ evtmon.vbs
http://blogs.technet.com/netmon/archive/2007/02/22/eventmon-stopping-a-capture-based-on-an-eventlog-event.aspx
‘======================================================================
‘ Print out the help when something is not typed in correctly or when
‘ nothing at all is typed in.

Public Sub PrintHelp
    Wscript.Echo “Usage:”
    Wscript.Echo ”  EvtMon EventNumber [LogFile]”
    Wscript.Echo ”    LogFile is optional.  If used, the eventlog name”
    Wscript.Echo ”    file ie, application, system, security, etc…”
End Sub

‘ Get the arguments.  Check for event nubmer and log file as arugments
Set objArgs = WScript.Arguments

‘ See how many arguments we have and colect them.
if objArgs.Count < 1 OR objArgs.Count > 2 Then
    PrintHelp
ElseIf objArgs.Count > 1 Then
    EventNumber = objArgs(0)
    LogFile = objArgs(1)
Else
    EventNumber = objArgs(0)
    LogFile = “”
End If

If EventNumber <> “” Then

    strComputer = “.”

    ‘ Attatch to the WMI Service
    Set objWMIService = GetObject(“winmgmts:{(Security)}\\” & _
            strComputer & “\root\cimv2”)

    ‘ if the LogFile is populated add this to our query.  Create a
    ‘ Event Log monitoring object and send it a query.
    If LogFile = “” Then
        Set colMonitoredEvents = objWMIService.ExecNotificationQuery _   
            (“Select * from __InstanceCreationEvent Where ” _
                & “TargetInstance ISA ‘Win32_NTLogEvent’ ” _
                    & “and TargetInstance.EventCode = ‘” _
                    & EventNumber & “‘”)
    Else
        Set colMonitoredEvents = objWMIService.ExecNotificationQuery _   
            (“Select * from __InstanceCreationEvent Where ” _
                & “TargetInstance ISA ‘Win32_NTLogEvent’ ” _
                    & “and TargetInstance.EventCode = ‘” _
                    & EventNumber _
                    & “‘ and TargetInstance.LogFile = ‘” _
                    & LogFile & “‘”)
    End If

    ‘ Create an object which returns when the next event occurs.
    Set objLatestEvent = colMonitoredEvents.NextEvent
    ‘ Print some info based on the event log we encountered.
    Wscript.Echo objLatestEvent.TargetInstance.User
    Wscript.Echo objLatestEvent.TargetInstance.TimeWritten
    Wscript.Echo objLatestEvent.TargetInstance.Message
    WScript.Echo objLatestEvent.TargetInstance.Logfile
    Wscript.Echo
End If

And the control batch file DUMPONEL.CMD, that does the rest of the work, also cannibalized from Paul’s post!

DUMPONEL.CMD:

@echo off
if “%1″==”” goto Usage
if “%2″==”” goto Usage

cscript //NoLogo EvtMon.vbs %2 %3
cscript adplus.vbs -hang -pn %1 -quiet

goto :EOF

:Usage
echo Usage:
echo   %0 ProcessName EventNumber Logfile
echo       Logfile is optional.  If used, the eventlog name
echo       file ie, applicaiton, system, security, etc…

So, for example:

I’ve put the two files in my Debugging Tools for Windows installation directory, and can use them with:

DUMPONEL w3wp.exe 2023

DumpOnEL runs EvtMon with the second and third arguments. When Evtmon returns (Evtmon just waits for the event it’s looking for, a bit like a conditional “pause” command), ADPlus runs and hang dumps everything with the first argument (process name) specified.

The process then continues running happily* after it’s finished being dumped, and the file ends up in a Hang_Mode_xxxxx folder under the current directory.