There are three different bugchecks related to Autoboost: Stop 0x153, Stop 0x162 and Stop 0x192. All of which are caused by slightly different error conditions.
Before we begin diving into these bugchecks, it’s imperative that you understand what AutoBoost is and how it aims to resolve common thread scheduling problems. AutoBoost was designed to overcome the priority inversion problem, which occurs when a higher priority level thread is blocked from running because a lower priority thread is holding a resource which the higher priority thread requires. This is a classic deadlock situation whereby two threads are preventing each other from processing. This is sometimes also known as CPU starvation.
The following excerpt from Windows Internals 7th Edition describes this problem:
Imagine the following situation: A priority 7 thread is running, preventing aWindows Internals 7th Edition Part 1
priority 4 thread from ever receiving CPU time. However, a priority 11 thread is
waiting for some resource that the priority 4 thread has locked. But because the
priority 7 thread in the middle is eating up all the CPU time, the priority 4 thread
will never run long enough to finish whatever it’s doing and release the resource
blocking the priority 11 thread. This scenario is known as priority inversion.
AutoBoost partly decides which threads to boost the priority for based on the locks which they hold and any locks which they may be waiting on too. This information is maintained in an array (currently maximum of 6) of _KLOCK_ENTRY objects. Each object describes the locks which the thread currently holds and any locks which the thread may be waiting upon itself.
8: kd> dt _KTHREAD -y LockEntries nt!_KTHREAD +0x320 LockEntries :  _KLOCK_ENTRY
The locks which are currently held by the thread are described by a red-black tree structure, this can be found in the lock entry object:
8: kd> dt _KLOCK_ENTRY -y OwnerTree nt!_KLOCK_ENTRY +0x030 OwnerTree : _RTL_RB_TREE
Additionally, the locks which the thread is waiting upon are held in another red-black tree:
8: kd> dt _KLOCK_ENTRY -y WaiterTree nt!_KLOCK_ENTRY +0x040 WaiterTree : _RTL_RB_TREE
The tree is then transversed to determine which priority level the thread should be boosted to. The owner thread’s priority is then boosted to the same priority as the thread which is waiting on the lock to be released. Please note that the thread’s priority will never be boosted beyond priority level 15. Additionally, the AutoBoost framework is currently only implemented for push locks and guarded mutexes.
You can dump push locks and mutexes respectively using the following structure:
3: kd> dt _EX_PUSH_LOCK nt!_EX_PUSH_LOCK +0x000 Locked : Pos 0, 1 Bit +0x000 Waiting : Pos 1, 1 Bit +0x000 Waking : Pos 2, 1 Bit +0x000 MultipleShared : Pos 3, 1 Bit +0x000 Shared : Pos 4, 60 Bits +0x000 Value : Uint8B +0x000 Ptr : Ptr64 Void
Guarded mutexes use the KGUARDED_MUTEX structure which isn’t available with public symbols.
Since a majority of these bugchecks appear to be caused by push locks, and this form of synchronisation mechanism is sparely documented, I thought it would be wise to add some information regarding their acquisition and release.
When you attempt to acquire a push lock, ExAcquirePushLockExclusiveEx will set-up a _KLOCK_ENTRY structure for the calling thread, it will then check if the push lock has been acquired by checking that the acquired bit has been set, and if so, then the lock entry will be added to the waiting tree of the corresponding lock entry. Otherwise, it will be added to the owner tree of the lock entry.
The LockState property of the _KLOCK_STATE_ENTRY structure is then set to the push lock which we’re acquiring.
9: kd> dt _KLOCK_ENTRY_LOCK_STATE -y LockState nt!_KLOCK_ENTRY_LOCK_STATE +0x000 LockState : Ptr64 Void
The ExfAcquirePushLockExclusiveEx function will be called if the push lock is already held by another thread; a _EX_PUSH_LOCK_WAIT_BLOCK structure is then constructed, which is the wait block for the push lock which we attempted to acquire. This wait block contains a doubly linked list of all the waiting threads which are waiting to acquire exclusive access to the push lock. It should be noted that push locks can not be acquired at DISPATCH_LEVEL (IRQL level 2) or above, attempting to do so, will likely end up in a Stop 0x192. However, please note, that for the thread to call ExAcquirePushLockExclusiveEx and attempt to grant exclusive access to the push lock, then the thread must enter a critical region which will raise the IRQL Level to 1 or APC_LEVEL, this then disables user-mode and normal kernel APCs.
To release exclusive access to a push lock, the ExReleasePushLockExclusive macro will be called, this will release the push lock and enable it to be acquired by another thread. If the 2nd bit of the push lock is set, then the ExfTryToWakePushLock is called which will set the gate object associated to the wait block to the signaled state, which will then cause any waiting thread to be released from it’s wait state, the push lock value will then be updated by using a lock cmpxchg instruction.
The lock entries will be checked for the thread which was holding the push lock by checking that the acquired bit has been set to true, and the lock state contains the same pointer as the push lock. Remember that push locks are merely pointers. If the push lock is unable to be found in any of the thread’s lock entries and the AutoBoostEntriesExhausted flag for the thread is false, then the system will bugcheck with a Stop 0x162.
The following video also provides some background on the acquisition and release of push locks:
The accompanying slides can be found in the references at the end of this post.
Now, we’ve gained some understanding of how Autoboost is implemented and the problem it aims to help to resolve. Let’s investigate the bugchecks which are associated to the AutoBoost framework. Firstly, we’ll examine the Stop 0x153, which indicates that a thread holding a lock was prematurely terminated.
KERNEL_LOCK_ENTRY_LEAKED_ON_THREAD_TERMINATION (153) A thread was terminated before it had freed all its AutoBoost lock entries. This is typically caused when a thread never released a lock it previously acquired (e.g. by relying on another thread to release it), or if the thread did not supply a consistent set of flags to lock package APIs. Arguments: Arg1: ffffd902d2aa7080, The address of the thread << Terminated Thread Arg2: ffffd902d2aa73a0, The address of the entry that was not freed << _KLOCK_ENTRY Arg3: 0000000000000001, Lock pointer was not NULL Arg4: ffffae08901abc50, The address of the lock associated with the entry << _EX_PUSH_LOCK
If we dump the thread object from the first parameter, we can indeed see, that the thread was terminated prematurely.
8: kd> !thread ffffd902d2aa7080 THREAD ffffd902d2aa7080 Cid 0f0c.1e64 Teb: 0000000000000000 Win32Thread: 0000000000000000 TERMINATED Not impersonating GetUlongFromAddress: unable to read from fffff8053162ba04 Owning Process ffffd902d2d6d080 Image: msiexec.exe Attached Process N/A Image: N/A fffff78000000000: Unable to get shared data Wait Start TickCount 444115 Context Switch Count 148757 IdealProcessor: 11 ReadMemory error: Cannot get nt!KeMaximumIncrement value. UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000778063b0 Stack Init 0000000000000000 Current ffffec8b4552c490 Base ffffec8b4552d000 Limit ffffec8b45527000 Call 0000000000000000 Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
It should be mentioned, that there is a number of AutoBoost fields on the _KTHREAD structure. I’ll mention the ones which are of most interest.
8: kd> dt _KTHREAD -y AutoBoostActive ffffd902d2aa7080 nt!_KTHREAD +0x074 AutoBoostActive : 0y1
The AutoBoostActive field either indicates if AutoBoost was enabled for the thread or it was AutoBoost tracking was enabled for the lock which the thread is holding; in either case, we can clearly see that it was enabled.
Additionally, there is another field called AutoBoostEntriesExhausted, while this was mentioned earlier, I’m not exactly sure of it’s purpose, but I presume it suggests if the thread has the maximum amount of lock entries?
8: kd> dt _KTHREAD -y AutoBoostEntriesExhausted ffffd902d2aa7080 nt!_KTHREAD +0x078 AutoBoostEntriesExhausted : 0y0
Lastly, there is the AbOwnedEntryCount which indicates the number of lock entries the thread currently owns. In this case there is only one and is this the one mentioned by the bugcheck description.
8: kd> dt _KTHREAD -y AbOwnedEntryCount ffffd902d2aa7080 nt!_KTHREAD +0x58f AbOwnedEntryCount : 0x1 ''
We know that the thread was still holding a lock when it was terminated. To investigate this further, let’s dump the associated lock entry object.
8: kd> dt _KLOCK_ENTRY ffffd902d2aa73a0 nt!_KLOCK_ENTRY +0x000 TreeNode : _RTL_BALANCED_NODE +0x000 FreeListEntry : _SINGLE_LIST_ENTRY +0x018 EntryFlags : 0x1010032 +0x018 EntryOffset : 0x32 '2' +0x019 ThreadLocalFlags : 0 '' +0x019 WaitingBit : 0y0 +0x019 Spare0 : 0y0000000 (0) +0x01a AcquiredByte : 0x1 '' +0x01a AcquiredBit : 0y1 +0x01b CrossThreadFlags : 0x1 '' +0x01b HeadNodeBit : 0y1 +0x01b IoPriorityBit : 0y0 +0x01b IoQoSWaiter : 0y0 +0x01b Spare1 : 0y00000 (0) +0x018 StaticState : 0y00110010 (0x32) +0x018 AllFlags : 0y000000010000000100000000 (0x10100) +0x01c SpareFlags : 0 +0x020 LockState : _KLOCK_ENTRY_LOCK_STATE +0x020 LockUnsafe : 0xffffae08`901abc50 Void +0x020 CrossThreadReleasableAndBusyByte : 0x50 'P' +0x021 Reserved :  "???" +0x027 InTreeByte : 0xff '' +0x028 SessionState : 0x00000000`ffffffff Void +0x028 SessionId : 0xffffffff +0x02c SessionPad : 0 +0x030 OwnerTree : _RTL_RB_TREE +0x040 WaiterTree : _RTL_RB_TREE +0x030 CpuPriorityKey : 0 '' +0x050 EntryLock : 0 +0x058 BoostBitmap : _KLOCK_ENTRY_BOOST_BITMAP +0x05c SparePad : 0
The AcquiredBit field is set, so we’re holding a lock and the LockUnsafe field points to the address of the push lock which we were holding at the time of the thread termination. Notice that it matches the address shown in the fourth parameter of the bugcheck?
8: kd> dt _EX_PUSH_LOCK ffffae08`901abc50 win32k!_EX_PUSH_LOCK +0x000 Locked : 0y1 +0x000 Waiting : 0y0 +0x000 Waking : 0y0 +0x000 MultipleShared : 0y0 +0x000 Shared : 0y000000000000000000000000000000000000000000000000000000000000 (0) +0x000 Value : 1 +0x000 Ptr : 0x00000000`00000001 Void
The Locked field is set, therefore we hold that a thread is currently holding this lock. As we know from examining the lock entry structure, this lock was held by our terminated thread. There isn’t much you can do to debug this bugcheck unfortunately, although, if we dump the call stack, we can see that the process which the thread was associated to had been terminated.
8: kd> knL # Child-SP RetAddr Call Site 00 ffffec8b`46c99588 fffff805`31431813 nt!KeBugCheckEx 01 ffffec8b`46c99590 fffff805`3185ec0e nt!KeCleanupThreadState+0x165887 02 ffffec8b`46c995d0 fffff805`317f8410 nt!PspThreadDelete+0x1e 03 ffffec8b`46c99640 fffff805`3123a969 nt!ObpRemoveObjectRoutine+0x80 04 ffffec8b`46c996a0 fffff805`317eb1b7 nt!ObfDereferenceObjectWithTag+0xc9 05 ffffec8b`46c996e0 fffff805`318366d5 nt!ObCloseHandleTableEntry+0x2c7 06 ffffec8b`46c99820 fffff805`31836371 nt!ExSweepHandleTable+0xd5 07 ffffec8b`46c998d0 fffff805`31836899 nt!ObKillProcess+0x35 08 ffffec8b`46c99900 fffff805`3185ceb3 nt!PspRundownSingleProcess+0x131 09 ffffec8b`46c99980 fffff805`31837f1b nt!PspExitThread+0x60b 0a ffffec8b`46c99a90 fffff805`313d2918 nt!NtTerminateProcess+0xeb 0b ffffec8b`46c99b00 00000000`777a1cbc nt!KiSystemServiceCopyEnd+0x28 0c 00000000`00f8ec48 00000000`00000000 0x777a1cbc
My suggestion would be to find out who is calling NtTerminateProcess. In our case, the process was msiexec.exe, which is a Windows installer used for installing .msi packages. The next bugcheck we’ll investigate is Stop 0x162, which when a thread has attempted to release a lock which it does not own.
KERNEL_AUTO_BOOST_INVALID_LOCK_RELEASE (162) A lock tracked by AutoBoost was released by a thread that did not own the lock. This is typically caused when some thread releases a lock on behalf of another thread (which is not legal with AutoBoost tracking enabled) or when some thread tries to release a lock it no longer owns. Arguments: Arg1: ffffdb8d223cb080, The address of the thread Arg2: ffffcb81844bab50, The lock address Arg3: 00000000ffffffff, The session ID of the thread Arg4: 0000000000000000, Reserved
The first parameter contains the address of the thread which attempted to release the lock. The first parameter contains the address of the thread which attempted to release the lock. The second parameter is the lock object in question. This appears to have been freed at the time of the crash. Let’s examine the call stack to see what may have happened. As we can see, it does certainly seem that a lock was released inappropriately. I’m not sure what exactly KeAbPostRelease does, however, judging from the function name, I would presume it’s some form of callback function which is called after a lock tracked by AutoBoost is released. I also noticed that the AbPostRelease function takes a parameter with the address of ffffcb81`844bab30, which is very close to the address of our push lock.
9: kd> knL # Child-SP RetAddr Call Site 00 ffff880e`4fb4f898 fffff804`281de4b5 nt!KeBugCheckEx 01 ffff880e`4fb4f8a0 fffff804`2867fb30 nt!KeAbPostRelease+0xcb5b5 << Crash here! 02 ffff880e`4fb4f8f0 fffff804`2867f355 nt!AlpcpReceiveMessage+0x620 << Lock released 03 ffff880e`4fb4f9d0 fffff804`281d3c15 nt!NtAlpcSendWaitReceivePort+0x105 04 ffff880e`4fb4fa90 00007fff`fb35d1a4 nt!KiSystemServiceCopyEnd+0x25 05 00000070`e20ff148 00000000`00000000 0x00007fff`fb35d1a4
We can find more information about what the lock is associated to by using the !pool command. It appears to be related to an ALPC message. This makes perfect sense since we were receiving an ALPC message at the time of the crash.
9: kd> !pool ffffcb81844bab50 Pool page ffffcb81844bab50 region is Paged pool ffffcb81844ba000 size: 580 previous size: 0 (Allocated) Ntff ffffcb81844ba590 size: 580 previous size: 0 (Allocated) Ntff *ffffcb81844bab20 size: 360 previous size: 0 (Allocated) *AlMs Pooltag AlMs : ALPC message, Binary : nt!alpc ffffcb81844bae80 size: 160 previous size: 0 (Free) ...}
The ALPC message object is stored at 0x40 bytes from the pool allocation start address. Let’s use the !alpc command with the message switch to dump the associated message.
9: kd> !alpc /m ffffcb81`844bab60 Message ffffcb81844bab60 MessageID : 0x19AC (6572) CallbackID : 0x40A40A (4236298) SequenceNumber : 0x0000016D (365) Type : LPC_REQUEST DataLength : 0x0054 (84) TotalLength : 0x007C (124) Canceled : No Release : No ReplyWaitReply : No Continuation : Yes OwnerPort : ffffdb8d14b8d880 [ALPC_CLIENT_COMMUNICATION_PORT] WaitingThread : 0000000000000000 QueueType : ALPC_MSGQUEUE_PENDING QueuePort : ffffdb8d14bb2c90 [ALPC_CONNECTION_PORT] QueuePortOwnerProcess : ffffdb8d14b6e440 (svchost.exe) ServerThread : ffffdb8d223cb080 << Same thread shown in the first parameter QuotaCharged : No CancelQueuePort : 0000000000000000 CancelSequencePort : 0000000000000000 CancelSequenceNumber : 0x00000000 (0) ClientContext : 0000017d1764ede0 ServerContext : 0000000000000000 PortContext : 00000265e5274040 CancelPortContext : 0000000000000000 SecurityData : ffffcb81803d9d70 View : 0000000000000000 HandleData : 0000000000000000
The actual lock object is stored as part of a larger structure called a blob. Blobs are special objects used by the ALPC library for managing objects related to ALPC such as messages. Blobs have similar attributes to objects exposed by the Object Manager: they can be garbage collected, have their references tracked and locked through synchronisation mechanisms such as push locks. Blobs are allocated using paged pool. We can dump a blob object by using the _BLOB structure.
9: kd> dt _BLOB ffffcb81844bab30 nt!_BLOB +0x000 ResourceList : _LIST_ENTRY [ 0xffffcb81`844bab30 - 0xffffcb81`844bab30 ] +0x000 FreeListEntry : _SLIST_ENTRY +0x010 u1 : <anonymous-tag> +0x011 ResourceId : 0x2 '' +0x012 CachedReferences : 0n0 +0x018 ReferenceCount : 0n1 +0x020 Lock : _EX_PUSH_LOCK << Second parameter
I believe the ResourceId is used to index into the _BLOB_ID enumeration; 0x2 points to a message type, which makes perfect sense since we’re dealing with a ALPC message.
9: kd> dt _BLOB_ID nt!_BLOB_ID BLOB_TYPE_UNKNOWN = 0n0 BLOB_TYPE_CONNECTION_INFO = 0n1 BLOB_TYPE_MESSAGE = 0n2 BLOB_TYPE_SECURITY_CONTEXT = 0n3 BLOB_TYPE_SECTION = 0n4 BLOB_TYPE_REGION = 0n5 BLOB_TYPE_VIEW = 0n6 BLOB_TYPE_RESERVE = 0n7 BLOB_TYPE_DIRECT_TRANSFER = 0n8 BLOB_TYPE_HANDLE_DATA = 0n9 BLOB_TYPE_MAX_ID = 0n10
Unfortunately, I haven’t been able to find much information at all about KeAbPostRelease. However, from reading some threads regarding this crash, it appears that this is commonly caused by drivers not implementing synchronization mechanisms correctly. I would strongly suggest running Driver Verifier with the deadlock verification checks enabled.
Lastly, there is the Stop 0x192, which indicates that a thread acquired a AutoBoost lock at an IRQL at DISPATCH_LEVEL or above.
KERNEL_AUTO_BOOST_LOCK_ACQUISITION_WITH_RAISED_IRQL (192) A lock tracked by AutoBoost was acquired while executing at DISPATCH_LEVEL or above. Arguments: Arg1: ffff990ec984a080, The address of the thread. Arg2: ffffa92601302f90, The lock address. Arg3: 0000000000000000, The IRQL at which the lock was acquired. Arg4: 0000000000000000, Reserved.
0: kd> dt nt!_KPRCB -y DebuggerSavedIrql fffff8074b809180 +0x5b98 DebuggerSavedIRQL : 0 ''
You may have noticed that the IRQL in the third parameter is at PASSIVE_LEVEL. I believe that this bugcheck has a generic umbrella description for all crashes which are related to threads attempting to acquire push locks. And the reason why the system crashes, is because the thread has already exhausted the number of locks which it can wait upon, the AutoBoostEntriesExhausted field is set to true in this instance, which is unlike the other Autoboost bugchecks which I’ve examined. I could be wrong, although, that is what makes the most sense to me in this circumstance.
0: kd> dt _KTHREAD -y AutoBoostEntriesExhausted ffff990ec984a080 nt!_KTHREAD +0x078 AutoBoostEntriesExhausted : 0y1
The win32kbase!GdiHandleEntryDirectory::AcquireEntryLock function does appear to enter and leave a critical region whilst attempting to obtain the push lock, this would only raise the IRQL Level to 1.
0: kd> !stack -p Call Stack : 12 frames ## Stack-Pointer Return-Address Call-Site 00 ffffd70155e14238 fffff8074ec39c35 nt!KeBugCheckEx+0 Parameter = 0000000000000192 Parameter = ffff990ec984a080 Parameter = ffffa92601302f90 Parameter = 0000000000000000 01 ffffd70155e14240 ffffa9528a5ea377 nt!ExAcquirePushLockExclusiveEx+145 (perf) Parameter = ffffa92601302f90 << Our push lock Parameter = 0000000000000000 Parameter = (unknown) Parameter = (unknown) 02 ffffd70155e14280 ffffa9528a5e82c4 win32kbase!GdiHandleEntryDirectory::AcquireEntryLock+c7 (perf) Parameter = (unknown) Parameter = ffff990ec984a080 << Thread attempting to acquire the lock Parameter = 0000000000000000 Parameter = (unknown)
Again unfortunately, I haven’t been able to gather much information from the dump file, and I would advise that Driver Verifier be run, it would be particularly interesting to see if the IRQL verification checks are able to catch anything.
Windows 10 Anniversary Update: GDI handle management and vulnerabilities exploitation
Pwning Windows Kernel – Unkillable User-Mode Process – Part 2
ReactOS: ntoskrnl/ex/pushlock.c File Reference
Windows Internals 6th Edition Part 1