/
Tracking file creation and access on Linux and Windows

Tracking file creation and access on Linux and Windows

This article explains how to identify which process created, modified, or accessed files on Linux and Windows to monitor file activity and trace system operations.

LAST TESTED ON CHECKMK 2.5.0P1

Table of Contents

Overview

When troubleshooting unexpected file creation, modification, or access on a Linux system, it is often necessary to determine which process is responsible for interacting with a file or directory.

This article explains how to use common Linux auditing and tracing tools to identify file activity, including:

  • auditctl and ausearch

  • inotifywait

  • strace

  • fatrace (optional)

These tools can be used during Checkmk investigations and general Linux troubleshooting to determine which process created, modified, or accessed a file.

 

Linux

Install Required Tools

Install the required packages as root or with sudo.

# auditd (includes auditctl, ausearch, aureport) sudo apt install auditd # inotifywait sudo apt install inotify-tools # strace sudo apt install strace # fatrace (optional, lightweight alternative) sudo apt install fatrace

 

Verify that the audit daemon is running:

sudo systemctl status auditd

 

If the service is not running:

sudo systemctl start auditd

 

 

Configure an Audit Watch

Audit rules allow Linux to record activity occurring within a specific directory.

In the example below, the Checkmk PDF generation directory is monitored:

auditctl -w /omd/sites/mysite/tmp/check_mk/pdf \ -p rwa \ -k lixu_all

 

Parameter explanation:

Description

Parameter

Description

Parameter

Path to monitor

-w

Permissions to watch

-p

Read operations

 r

Write operations

 w

Attribute changes

 a

Execute operations

 x

Search key used later with ausearch

-k

 

Verify the Rule

List active audit rules:

auditctl -l

 

Remove a Duplicate Rule

If a duplicate watch exists, remove it using the same parameters:

auditctl -W /omd/sites/mysite/tmp/check_mk/pdf -p rwa -k lixu_all

Use an uppercase -W when removing a watch rule.

 


Search Audit Logs

Once activity has occurred, use ausearch to identify the responsible process.

 

Display All Events

ausearch -k lixu_all -i

The -i option converts IDs and syscall numbers into human-readable output.

 

Search for a File and Specific System Call

ausearch -k lixu_all -f tmpvxgg_s94 --syscall openat -i

 

Find File Creation Events

To identify which process created a file:

ausearch -k lixu_all -i | grep -B10 "nametype=CREATE"

 

Filter by Executable

For example, to view activity generated by Apache:

ausearch -k lixu_all -x /usr/sbin/apache2 -i

 

Filter by PID

ausearch -k lixu_all --pid 1344368 -i

 

Filter by Parent PID

ausearch -k lixu_all --ppid 1141701 -i

 

Filter by Time Range

ausearch -k lixu_all --start 18:40:00 --end 18:45:00 -i

 

 

Monitor File Activity in Real Time

inotifywait provides immediate notification when files appear or are moved into a directory.

Unlike auditd, inotifywait does not identify the process responsible for the action. Use the timestamp together with ausearch to determine the originating process.

 

Watch for File Creation

inotifywait -m /omd/sites/mysite/tmp/check_mk/pdf \ -e create -e moved_to

 

Watch with Timestamps

inotifywait -m -r \ --timefmt '%H:%M:%S' \ --format '%T %w%f %e' \ -e create -e moved_to \ /omd/sites/mysite/tmp/check_mk/pdf

 

Filter for PDF and Temporary Files

inotifywait -m /omd/sites/mysite/tmp/check_mk/pdf \ -e create 2>&1 | grep -E "\.pdf|tmp"

 

 

Trace File Operations with strace

For deeper analysis, strace can capture the exact system calls used by a process.

This is particularly useful when investigating application behavior.

 

Attach to All Apache Workers

strace $(pgrep -f "apache2" | sed 's/^/-p /') \ -e trace=openat,creat -f \ -o /tmp/strace_apache.log

 

Parameter explanation:

Description

Parameter

Description

Parameter

Follow child processes

-f

Write output to a file

-o

Record file creation and open operations

trace=openat,creat

 

Monitor Activity Live

tail -f /tmp/strace_apache.log | grep "check_mk/pdf"

 

Search the Saved Log

Find file creation operations:

grep "O_CREAT" /tmp/strace_apache.log

 

Search for a specific directory:

grep "check_mk/pdf" /tmp/strace_apache.log

 

 

Example Investigation Result

During a Checkmk PDF reporting investigation, the following pattern was observed:

  • An Apache worker process created a temporary file using O_CREAT|O_EXCL.

  • A child pdftoppm process opened the file shortly afterward using O_RDONLY.

  • The file was used to generate a PNG preview for the report.

This behavior is expected during PDF report generation and does not indicate a problem.

 

 

Cleanup After the Investigation

Audit rules and logs should be removed after troubleshooting is complete to avoid unnecessary disk usage.

 

Remove the Audit Watch

auditctl -W /omd/sites/mysite/tmp/check_mk/pdf -p rwa -k lixu_all

 

Verify removal:

auditctl -l

 

Rotate Audit Logs

service auditd rotate

 

Remove Old Audit Logs

rm -f /var/log/audit/audit.log.*

 

Check Current Audit Log Size

du -sh /var/log/audit/

 

Stop auditd (Optional)

If the service was enabled only for troubleshooting:

systemctl stop auditd systemctl disable auditd

 

 

Validation

To confirm that auditing is functioning correctly:

  1. Create a test file within the monitored directory.

  2. Verify that an audit event is generated.

  3. Search the audit log using the configured key.

  4. Confirm that the responsible process appears in the results.

 

Example:

touch /omd/sites/mysite/tmp/check_mk/pdf/testfile.txt ausearch -k lixu_all -i

The output should contain a CREATE event associated with the process that generated the file.

 

 

Troubleshooting

No Audit Events Are Recorded

Verify that auditd is running:

systemctl status auditd

 

Confirm that the watch rule exists:

auditctl -l

 

inotifywait Shows Events but No Process Information

This is expected behavior.

Use the event timestamp and correlate it with audit data:

ausearch -k lixu_all --start HH:MM:SS -i

 

strace Does Not Show File Operations

Verify that:

  • The correct process is being traced.

  • The process is actively generating file activity.

  • Child processes are being followed using the -f option.

 

Windows

All commands in this section must be run from an elevated Command Prompt or PowerShell session.

Windows includes native auditing tools that can be used to determine which process created, modified, deleted, or accessed files. While the workflow differs from Linux, the overall investigation process is similar:

  1. Enable auditing.

  2. Configure monitoring on the target folder.

  3. Reproduce the issue.

  4. Review audit logs.

  5. Correlate events to the responsible process.

  6. Remove auditing when finished.

 

Enable File System Auditing

Windows file auditing is configured using auditpol.exe.

Enable auditing for both successful and failed file system operations:

auditpol.exe /set /subcategory:"File System" /success:enable /failure:enable

 

Verify the setting:

auditpol.exe /get /subcategory:"File System"

This enables Windows to record file access events in the Security Event Log.

 

 

Configure Auditing on a Folder

After enabling auditing globally, configure the specific folder you want to monitor.

 

Using the Command Line

The following example configures auditing on a folder:

icacls "C:\path\to\folder" /grant Everyone:(OI)(CI)(S:W,Cr,R)

 

Description

Parameter

Description

Parameter

Object Inherit (files)

OI

Container Inherit (subfolders)

CI

Write operations

W

File creation

Cr

Read operations

R

 

Using the Windows GUI

Alternatively:

  1. Right-click the target folder.

  2. Select Properties.

  3. Open the Security tab.

  4. Select Advanced.

  5. Open the Auditing tab.

  6. Add the users or groups to audit and select the desired actions.

 

Search the Security Event Log

Windows records file activity in the Security Event Log.

Common event IDs include:

Description

Event ID

Description

Event ID

Handle requested

4656

Object accessed

4663

Object deleted

4660

 

Display All File Access Events

wevtutil qe Security /q:"*[System[EventID=4663]]" /f:text /rd:true

 

Search for a Specific File

wevtutil qe Security /q:"*[System[EventID=4663] and EventData[Data[@Name='ObjectName'] and Data='C:\path\to\folder\myfile.tmp']]" /f:text

 

Search Within a Time Range

wevtutil qe Security /q:"*[System[EventID=4663 and TimeCreated[@SystemTime>='2024-01-01T18:40:00Z' and @SystemTime<='2024-01-01T18:45:00Z']]]" /f:text

The /f:text option converts the output into a human-readable format similar to using ausearch -i on Linux.

 

 

Monitor File Activity in Real Time

Microsoft Sysinternals Process Monitor (Procmon) provides real-time visibility into file operations and is the closest Windows equivalent to a combination of inotifywait and strace.

 

Process Monitor can show:

  • Which process accessed a file

  • The exact operation performed

  • Success or failure results

  • Timestamps for every operation

 

Capture Activity from the Command Line

Start recording:

Procmon.exe /Quiet /Minimized /BackingFile C:\tmp\procmon.pml

 

Allow the issue to occur, then stop the capture:

timeout 30 Procmon.exe /Terminate

 

Export the capture:

Procmon.exe /OpenLog C:\tmp\procmon.pml /SaveApplyFilter /SaveAs C:\tmp\out.csv

 

Search the results:

findstr /i "C:\path\to\folder" C:\tmp\out.csv

 

 

PowerShell Audit Scripts

For larger investigations, PowerShell scripts can simplify setup and cleanup.

 

Setup Audit

Use this script before reproducing the issue.

 

The script:

  • Enables File System auditing

  • Configures the target folder's SACL

  • Verifies the configuration

# setup_audit.ps1 # Enables Windows file access auditing for a target folder # Run as Administrator # --- CONFIG --- $TargetPath = "C:\ProgramData\checkmk" # -------------- # 1. Enable File System auditing via auditpol Write-Host "Enabling File System auditing policy..." -ForegroundColor Cyan auditpol.exe /set /subcategory:"File System" /success:enable /failure:enable # 2. Set SACL on the folder Write-Host "Setting SACL on $TargetPath..." -ForegroundColor Cyan $acl = Get-Acl $TargetPath $rule = New-Object System.Security.AccessControl.FileSystemAuditRule( "Everyone", "Write,ReadData,CreateFiles", "ContainerInherit,ObjectInherit", "None", "Success" ) $acl.AddAuditRule($rule) Set-Acl $TargetPath $acl # 3. Verify Write-Host "`nVerifying auditpol policy..." -ForegroundColor Cyan auditpol.exe /get /subcategory:"File System" Write-Host "`nVerifying SACL on $TargetPath..." -ForegroundColor Cyan (Get-Acl $TargetPath -Audit).Audit | Format-List Write-Host "Audit is now active on $TargetPath" -ForegroundColor Green Write-Host "Let the system run, then execute teardown_audit.ps1 to export and clean up." -ForegroundColor Yellow

 

Example usage:

powershell -ExecutionPolicy Bypass -File .\setup_audit.ps1

 

 

Teardown Audit

Use this script after collecting data.

 

The script:

  • Exports all Event ID 4663 entries

  • Disables File System auditing

  • Removes the SACL from the folder

  • Creates a log file that can be provided to support

# teardown_audit.ps1 # Exports the audit log, disables auditing, and removes the SACL # Run as Administrator when investigation is complete # --- CONFIG --- $TargetPath = "C:\ProgramData\checkmk" $OutputFile = "C:\tmp\security_events.txt" # -------------- # 1. Export events to log file Write-Host "Exporting security events to $OutputFile..." -ForegroundColor Cyan $outDir = Split-Path $OutputFile if (!(Test-Path $outDir)) { New-Item -ItemType Directory -Path $outDir | Out-Null } wevtutil qe Security /q:"*[System[EventID=4663]]" /f:text /rd:true > $OutputFile Write-Host "Exported to $OutputFile" -ForegroundColor Green # 2. Disable File System auditing Write-Host "`nDisabling File System auditing policy..." -ForegroundColor Cyan auditpol.exe /set /subcategory:"File System" /success:disable /failure:disable # 3. Remove the SACL from the folder Write-Host "Removing SACL from $TargetPath..." -ForegroundColor Cyan $acl = Get-Acl $TargetPath $acl.RemoveAuditRuleAll((New-Object System.Security.AccessControl.FileSystemAuditRule( "Everyone", "Write,ReadData,CreateFiles", "ContainerInherit,ObjectInherit", "None", "Success" ))) Set-Acl $TargetPath $acl # 4. Verify SACL is gone Write-Host "`nVerifying SACL removed..." -ForegroundColor Cyan $remaining = (Get-Acl $TargetPath -Audit).Audit if ($remaining) { $remaining | Format-List } else { Write-Host "No audit rules remaining." -ForegroundColor Green } Write-Host "`nDone. Send $OutputFile for analysis." -ForegroundColor Green

 

Example usage:

powershell -ExecutionPolicy Bypass -File .\teardown_audit.ps1

 

Based on the script, the exported log is written to:

C:\tmp\security_events.txt

 

 

Cleanup After the Investigation

After the investigation is complete, remove the auditing configuration to prevent unnecessary event generation.

 

Disable File System Auditing

auditpol.exe /set /subcategory:"File System" /success:disable /failure:disable

 

Remove Folder Auditing

icacls "C:\path\to\folder" /remove:s Everyone

 

Export and Clear the Security Log (Optional)

Export the log:

wevtutil epl Security C:\tmp\security_backup.evtx

 

Clear the log:

wevtutil cl Security

 

 

Validation

To verify that auditing is functioning correctly:

  1. Enable auditing.

  2. Configure the target folder.

  3. Create or modify a test file.

  4. Query Event ID 4663 entries.

  5. Confirm that the event references the expected file and process.

 

Successful events should appear in the Security Event Log and provide sufficient information to identify the responsible application or user.

 


Related articles