Introduction
How to write code to detect when any software application on the machine accesses any file on the machine, and how to extract and view any data read from or written to that file in your own application, in real time, using C# and a little bit of C++?
That's the subject of today's article: how to build a file monitor on steroids (or a rough draft of one).
In the above screen shot, we're monitoring the Yahoo Messenger application. And as you can see, it reads and writes quite a bit of data. Public keys, log file snippets, some XML filtering stuff, and so forth. We could have as well chosen to point the tool at Microsoft Word, Grand Theft Auto, or the humble Notepad.exe. But the real power of these techniques is incorporating them into your own applications.
For example, those of you who've been following the online poker botting series can use these exact methods to detect and respond to log file and poker hand history text in real time, as its generated by the poker client. Let's take our tool and point it at the Poker Stars client, POKERSTARS.EXE, by way of example:
Sure enough, we've detected that Poker Stars has opened the log file, and sure enough, we're reading and displaying the log file data as it appears. Which is almost exactly what we'd like our poker bot to do.
But maybe poker's not you're thing. Maybe you're just tired, as I am, of third-rate software treating your hard drive like one of these:
Modern software apps are footloose and fancy-free when it comes to depositing cruft on your machine. I mean the temporary files that never get deleted. The hidden cookies that persist until you reinstall the operating system. And the registry entries which don't get culled when the application that owns them is uninstalled.
Fighting Back
It's a widely-known fact that file I/O on Windows systems is a matter of public record. It's public because, with admin-level access, you yourself can browse to any file on the machine, open the file, edit it, and save it. It's public because you have access to the process address space of every application on the machine (with few exceptions). It's public because applications that read from or write to files do so using a small set of publically available and thoroughly documented functions:
- CreateFile
- WriteFile
- ReadFile
- CloseHandle
- And a few others
The above functions are formal members of the exclusive country club known as the Windows API. They're called by millions of applications around the world every day. As you're reading this, various applications on your machine are calling these functions, whether they know it or not and whether you know it or not. In fact, it's hard to do much of anything on a Windows box without calling one of the above functions, either directly or indirectly.
But I'm a .NET programmer! you say. I'm a Java programmer! I don't talk to low-level Windows APIs!
Ah, but you do. You won't call these functions directly, but the .NET framework, the JVM implementation, or whatever library or framework you're using will. So the above set of file-manipulation APIs constitute a sort of bottleneck through which a majority of system file I/O must pass. Now: what if we could somehow get Windows to call one of our functions every time a particular application (any application) calls or causes to be called one of the above APIs?
File monitoring would be a cinch in that case. And that's essentially what we're going to do.
Creating the Injection DLL
The above application was written in C#, but it works in conjunction with a small C++ "workhorse" DLL. Inside this DLL, we'll code custom but equivalent versions of whatever Windows APIs we're interested in redirecting. Here's an example showing our custom version of CreateFile, dubbed "Mine_CreateFile" but you can call it whatever you want:
// return type, and calling convention as the version of CreateFile provided by the OS.
HANDLE WINAPI Mine_CreateFile(LPCWSTR lpFileName,DWORD dwDesiredAccess,
DWORD dwShareMode, LPSECURITY_ATTRIBUTES lpSecAttr,
DWORD dwCreateDisp, DWORD dwFlagsAttr,HANDLE hTemplate)
{
// First, call the original CreateFile provided by the operating system.
HANDLE hFile = Real_CreateFile(lpFileName,dwDesiredAccess,dwShareMode,lpSecAttr,
dwCreateDisp,dwFlagsAttr,hTemplate);
// Now, do whatever we want with the filename (lpFileName)
// Now, do whatever we want with the file handle (hFile)
// Return the same value returned to us by the original API
return hFile;
}
Pretty straightforward. Whenever this function is called, we call the original CreateFile API through the Real_CreateFile function pointer. Once that's done, we can snoop on the parameters (such as the filename) passed to this function by the target application, report them, etc. Now, in order for this code to work, we have to ensure one thing: our code must be running inside the target application's process.
A New Flavor of DLL Injection
I've written about DLL injection before, and there's a lot of information on this topic scattered around the net and in books. So I'll assume you know that DLL Injection is robust and reliable. I'll assume you know it's a carefully-designed operating system facility, not some backdoor hack. And I'll assume you know (or can figure out) how to inject a DLL. After all, it's a one-liner:
For the purpose of monitoring file I/O in real time, any DLL injection technique will work. But there's one which is particularly well-suited for this kind of thing. I mean the DetourCreateProcessWithDLL function, provided by the Microsoft Detours library:
STARTUPINFO si;
PROCESS_INFORMATION pi;
ZeroMemory(&si, sizeof(si));
ZeroMemory(&pi, sizeof(pi));
si.cb = sizeof(si);
// The following call loads the target executable with our DLL attached
BOOL bSuccess = DetourCreateProcessWithDll( targetExePath, NULL, NULL, NULL, TRUE,
CREATE_DEFAULT_ERROR_MODE | CREATE_SUSPENDED, NULL, NULL,
&si, &pi, "detoured.dll", "XFileMonitor.Hook.dll", NULL);
// DetourCreateProcessWithDll runs the process in a suspended state. Wake it up.
if (bSuccess)
ResumeThread(pi.hThread);
The benefit of doing it this way, as opposed to Windows Hooks or other methods of DLL injection? It ensures that our DLL will be present during the target application's startup routine, which is traditionally when programmers open a lot of files. Were we to inject using a Windows Hook, we'd miss application startup every time, because it takes several seconds for a Windows Hook DLL to be propagated across the system.
Detouring the System File APIs
It's not enough to simply create a DLL and inject it into the target application. We have to somehow redirect calls to the system-provided file APIs to the equivalent versions we've created. That's easily accomplished, using Microsoft Detours and a few lines of code.
{
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
if (::GetModuleHandle(L"XFILEMONITOR.GUI.EXE") == NULL)
{
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourAttach(&(PVOID&)Real_CreateFile, Mine_CreateFile);
DetourAttach(&(PVOID&)Real_CloseHandle, Mine_CloseHandle);
DetourAttach(&(PVOID&)Real_WriteFile, Mine_WriteFile);
DetourAttach(&(PVOID&)Real_ReadFile, Mine_ReadFile);
DetourTransactionCommit();
}
break;
case DLL_THREAD_ATTACH: break;
case DLL_THREAD_DETACH: break;
case DLL_PROCESS_DETACH:
if (::GetModuleHandle(L"XFILEMONITOR.GUI.EXE") == NULL)
{
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourDetach(&(PVOID&)Real_CreateFile, Mine_CreateFile);
DetourDetach(&(PVOID&)Real_CloseHandle, Mine_CloseHandle);
DetourDetach(&(PVOID&)Real_WriteFile, Mine_WriteFile);
DetourDetach(&(PVOID&)Real_ReadFile, Mine_ReadFile);
DetourTransactionCommit();
}
break;
}
return TRUE;
}
C and C++ programmers will recognize this is as a standard DllMain function. The DLL_PROCESS_ATTACH notification tells us that this DLL is being mapped into a new process - in this case, the process of the target application we'd like to spy on. That's as good a place as any to perform the redirection. The following lines of code...
DetourAttach(&(PVOID&)Real_CloseHandle, Mine_CloseHandle);
DetourAttach(&(PVOID&)Real_WriteFile, Mine_WriteFile);
DetourAttach(&(PVOID&)Real_ReadFile, Mine_ReadFile);
...cause all calls to the original versions of the function (the ones provided by the operating system) to be redirected to our custom versions of those functions. It happens transparently, behind the scenes, and in memory - you're not actually changing the system DLLs as they exist on disk, so you don't have to worry about corruption. And of course, when the DLL is unloaded from the process (DLL_PROCESS_DETACH) we remove the detours.
Simple.
Spying on the Birth and Death of Files
So you've got your DLL containing custom versions of various Windows APIs. You've injected your DLL into the target process using DetourCreateProcessWithDLL or your injection method of choice. And you've redirected all calls to various system APIs such that your code gets called instead. Now it's time to start tracking files as they're opened and closed by the target application. Let's revisit our CreateFile override, and add a little bit of code to do this:
map<HANDLE, CString> g_openFiles;
// Critical section guards access to the above collection across threads.
// Elsewhere in code we've called InitializeCriticalSection
CRITICAL_SECTION g_CritSec;
// Our custom version of the CreateFile API.
HANDLE WINAPI Mine_CreateFile(LPCWSTR lpFileName,DWORD dwDesiredAccess,
DWORD dwShareMode, LPSECURITY_ATTRIBUTES lpSecAttr,
DWORD dwCreateDisp, DWORD dwFlagsAttr,HANDLE hTemplate)
{
// First, all the original CreateFile provided by the operating system.
HANDLE hFile = Real_CreateFile(lpFileName,dwDesiredAccess,dwShareMode,lpSecAttr,
dwCreateDisp,dwFlagsAttr,hTemplate);
// If that was successful... a new file handle has been born
if (lpFileName && hFile)
{
CString sFileName = lpFileName;
if (!sFileName.IsEmpty())
{
// Store it! Multiple threads may call this function at the
// same time, we we'll use a critical section to ensure that
// only one of them manipulates g_openFiles at a given time.
::EnterCriticalSection(&g_CritSec);
g_openFiles.insert(pair<HANDLE, CString>(hFile, sFileName));
::LeaveCriticalSection(&g_CritSec);
}
}
return hFile;
}
Every time the target application calls (or indirectly causes to be called) CreateFile, we're storing the filename and the HANDLE associated with it. You could also take this opportunity to dump some information in a log file or (in the case of the XFileMonitor) display a notification in your GUI (which is running in a different process) that a particular file has been opened.
Snooping on File Reads and Writes
Similar to the custom version of CreateFile above, we'll create custom versions of the WriteFile and ReadFile APIs. The target application will call our versions of these APIs whenever it tries to read data from or write data to the disk. Let's take a look at our version of ReadFile.
LPDWORD lpNumberOfBytesRead, LPOVERLAPPED lpOverlapped)
{
// Call the original version of ReadFile provided by the OS
BOOL bSuccess = Real_ReadFile(hFile, lpBuffer, nNumberOfBytesToRead,
lpNumberOfBytesRead, lpOverlapped);
// See if the handle being written to is one we know about
map<HANDLE, CString>::const_iterator iter = g_openFiles.find(hFile);
if (iter != g_openFiles.end())
{
// Yes, we collected this handle in an earlier call to CreateFile.
// Retrieve the file name and do something with it.
CString fileName = (*iter).second;
// TODO: lpBuffer now contains whatever data the application read. This
// might be text data (either single-byte or Unicode) or it might be
// binary.. we're free to examine and/or transmit it as we see fit
}
return bSuccess;
}
First we pass the call along to the original API. Then we take the supplied HANDLE and see if it's in the list of handles we created by monitoring calls to CreateFile. If it is, we can easily get the associated filename. If it's not, we can still get the filename, but it's a bit more work, and left as a reader exercise.
Identifying Binary vs. Text Data the Quick and Dirty Way
If you're targeting or trying to monitor a particular file or application, such as monitoring the log file created by the Poker Stars gaming client, you already presumably know about the files you want to snoop on. You know for example, that hand history files are text files. So when you detect a write to a hand history file, you can be sure it's text data that's being written.
But what about when you don't know anything about the files? What if you're trying to write a generic file monitor, such as the one shown above?
The problem is that ReadFile and WriteFile treat everything as a buffer of bytes. It might be text data, that's being written. Then again, it might be binary data. You don't really know, and there's no quick and easy way to tell the difference, because binary data and text data will frequently use some of the same underlying values. For example, if the first byte of data in the buffer has the value 65, it might be the letter "A" (which, in its single-byte form, has the value 65) or it might be that you're reading a binary file, and the 65 indicates something (anything) else entirely. Maybe it means there are 65 records in this particular file, or maybe it means that the average price of a pound of bok choi in Moscow is 65 pennies. You don't know, I don't know. Only the person who wrote the code knows.
So what to do in that case?
Well, we can get a quick-and-dirty approximation of whether a given buffer of data is text or binary by:
- Checking the file extension. This isn't always reliable, since you can give any file, whether text or binary, any extension. But it's usually reliable.
- Checking the data being written to see if it falls within normal ranges for text character data. Again, not always reliable, but often reliable.
You already know how to get the file name (by detouring CreateFile). So let's look at some quick-and-dirty code to scan a particular buffer and see if the data in that buffer can successfully be interpreted as ASCII text. Specifically, let's check each byte in the buffer to see if it falls within normal ranges for ASCII (single-byte) character data. Here's a sample function which does that:
// if they fall within normal printable ranges for ASCII/MBCS
// character data.
bool IsAsciiText(LPCSTR buffer, int testLength)
{
int validChars = 0;
for (int index = 0; index < testLength; index++)
{
char c = buffer[index];
// if the value of C is negative, or if it
// identifies a non-printable ASCII character,
// then this is probably binary data.
if (char < 0 || !(isprint(c) || isspace(c)))
return false;
}
// Every character we tested was within the range
// for printable ASCII characters. This is probably
// text data.
return true;
}
Now, this little function is far from complete. It only handles ASCII (single-byte) character text. It doesn't correctly handle embedded NULLs and so forth. But it's good enough for demonstration purposes and pretty easy to enhance with additional, smarter checks.
Transmitting File Read and Write Data to Managed Applications
So you've got your ReadFile and WriteFile detours in place and working. You're able to roughly detect when particular data is text or binary, by examining the file name as well as the data stream itself. Now how do you go about transmitting all this data back to your application, running in a separate process?
Obviously we'll have to use some sort of inter-process communication. The exact flavor is up to you, but the above XFileMonitor tool uses WM_COPYDATA because it's simple and straightforward.
struct FILECONTEXT
{
HANDLE File; // the handle of the file read/written
int OriginalAPI; // a number uniquely identifying the specific API
// (such as ReadFile, etc) that was called.
};
// Transmit file read/write data back to the XFileMonitor (or any) application..
void Transmit(Win32API sourceAPI, HANDLE hFile, LPCWSTR text)
{
// We're using WM_COPYDATA, which is a window message, so we need a target window.
HWND hWnd = ::FindWindow(NULL, L"Coding the Wheel - XFileMonitor v1.0");
if (!hWnd)
return;
// Set up the COPYDATASTRUCT expected by Windows.. the length should be the
// size of our (custom) FILECONTEXTstructure, plus the length of the file
// data we're sending, expressed in bytes, with enough room for a terminating
// NULL.
COPYDATASTRUCT cds;
::ZeroMemory(&cds, sizeof(COPYDATASTRUCT));
cds.dwData = action;
cds.cbData = sizeof(FILECONTEXT) + ((wcslen(text)+1) * 2);
// Allocate the outgoing array
LPBYTE pOutData = new BYTE[cds.cbData];
// Place a FILECONTEXT structure at the front of this array
FILECONTEXT ht;
ht.File = hFile;
ht.OriginalAPI = sourceAPI;
memcpy(pOutData, &ht, sizeof(FILECONTEXT));
// Place the text immediately following the structure..assumes any
// single-byte text has already been converted to Unicode
wcscpy((LPWSTR)(pOutData + sizeof(FILECONTEXT)), text);
// Send it off
cds.lpData = pOutData;
::SendMessage(hWnd, WM_COPYDATA, (WPARAM)::GetDesktopWindow(), (LPARAM)&cds);
delete [] pOutData;
}
Then in our managed code application .EXE, we can use some simple P/Invoke to access the data. First, let's override OnWndMessage:
const int WM_COPYDATA = 0x4A;
// Get access to the WM_COPYDATA message
protected override void WndProc(ref Message m)
{
if (m.Msg == WM_COPYDATA)
OnCopyData(ref m);
base.WndProc(ref m);
}
Now let's define the OnCopyData function. In order to do that, we'll need to create P/Invoke compatible versions of the COPYDATASTRUCT (defined by windows) and FILECONTEXT (defined by us) structures.
private struct COPYDATASTRUCT
{
public int dwData;
public int cbData;
public IntPtr lpData;
};
// Managed version of FILECONTEXT struct (defined by us)
private struct FILECONTEXT
{
public IntPtr Handle;
public System.Int32 OriginalAPI;
}
// Decode WM_COPYDATA sent from the unmanaged C++ file monitoring DLL
private void OnCopyData(ref Message m)
{
COPYDATASTRUCT cds = new COPYDATASTRUCT();
cds = (COPYDATASTRUCT)Marshal.PtrToStructure(m.LParam, typeof(COPYDATASTRUCT));
FILECONTEXTht = (FILECONTEXT)Marshal.PtrToStructure(cds.lpData, typeof(FILECONTEXT));
string strBufferData;
int unManagedSize = Marshal.SizeOf(typeof(FILECONTEXT));
unsafe
{
byte* pString = (byte*)cds.lpData.ToPointer();
pString += unManagedSize;
IntPtr pManString = new IntPtr((void*)pString);
strBufferData = Marshal.PtrToStringUni(pManString);
// TODO: strBufferData contains the text sent over by the
// file monitor. Display it, analyze it, whatever.
}
}
And that's all there is to it.
I had very little time to throw this code together so I hope you'll excuse some of its obvious warts such as the use of unsafe C# code. Think of it as a quick and dirty example of techniques rather than production botting code. This code will be extended in Part 8 of this series (already written, and due shortly) in a direction you might not have guessed...
Putting It All Together: the XFileMonitor Application
You've seen what it looks like. You've read about the techniques behind it. Time to download the thing and test it out.
The source code contains four projects:
- An EXE project, implemented in C#
- A DLL project, implemented in C++
- 2 Detours projects: detours.dll and detoured.dll.
These projects boil down to essentially two files, which contain all the source code demonstrated above. In addition, there are two projects needed by the Detours Library which you can mostly ignore. Boost is not required for this project, and neither is NMAKE. So those of you that had trouble building the FoldBot will hopefully find this project is a cleaner build.
And as usual, my standard source-code disclaimer applies!
Good luck, and enjoy.
Posted by James Devlin 59 comment(s)
