Installing SPASS


SPASS – System Performance and Sizing Site was designed to be both easy to setup and maintain. We utilized only standard Windows .net components to keep the installation straightforward and decided to use a simple File Share for data storage as opposed to a SQL Database for both ease of installation as well as make it pretty much maintenance-free.

There are 3 main components that comprise the SPASS system:

  1. The Data Store where all of the Performance Data is stored. This is a simple file share (or multiple file shares) that is accessed by both the website and the data collectors.
  2. The Data Collectors which collect the data. These can be ran either locally on each server or can be ran from a central server that collects the data remotely from the other systems.
  3. The Website which presents the collected data for a group a servers that you specify (we refer to this as an “Application Group”). For larger installs, there multiple “Application Group” sites as well as a main site that lists all of the Application Groups as well as all the monitored servers.

We are also working on additional applications that will utilize the collected data to create professional and advanced Performance Reports as well as the option to also store the collected performance data in a SQL Database so you can create your own reports.

System Requirements

Data Collector System Requirements

Network Data Collector System Requirements

The Network Data Collector will remotely collect data from Windows Server ® 2003 and above computers

  • Windows Server ® 2008R2 or above
  • Microsoft .net 4.0
  • Remote Systems must be collecting Perfmon Data in BLG Files
  • The user account the Network Collectors runs as must have access to read the BLG files and pull WMI and Log info from the remote systems

Web Application System Requirements

  • IIS with a .net 4 Application Pool (IIS 7 or above is recommended)
  • “Virtual Directory” Pointing to the Data Store where the collected Performance Data is located

Setting Up the Data Store / File Share

SPASS can use either a single Windows Share to store the data, or 2 Shares, one for the “current” statistics (stats from the last 3-4 hours) and one for the hourly and daily statistics.

We present the option of separating out the “current” statistics if you want to store these stats directly on the web server and the hourly/daily stats on a remote share which may have more storage space. Another consideration is that the “landing pages” of the SPASS Website uses the current stats, so it might also improve site performance if the current stats are stored directly on the Web Server that hosts the websites.

For Disk Space requirements, the “current stats” will not change too much and usually only require well under 1MB per server. The hourly/daily stats usually require around 250-300MB of storage space for each month of collected data.

Tip: SPASS was written to utilize the “Daily Stats” for the Weekly and Monthly pages, so if space is a concern, you can minimize disk space by removing old “Hourly Stats” that may no longer be needed and still benefit from viewing the system’s Weekly and Monthly Stats. If you remove the Hourly Stats, the server storage usage drops to about 10-15MB per server per month. We include a sample PowerShell script that you can reference if you need to remove older hourly stats and leave the daily stats due to storage space concerns.

The permissions of these shares require any account that will run the Data Collector applications be able to write to the share. Usually the Local Data Collectors run as the System account and the Network Data Collectors require an account that has Admin permissions on the remote servers. Also to be able to access the images via the web front-end, the account that the app pool runs as needs read access to the share and the files stored there.

A common share configuration is to allow the “Domain Computers” and “Domain Administrators” full access to the share and everyone else read-only permissions.

Setting Up the Data Collector

The Data Collector Applications were created to easily be rolled out into your organization using standard Windows functions. Apart from copying the files, the “config.exe” application will aid you in configuring the programs as well as setup Scheduled Tasks to periodically run the collectors.

Tip: An example .bat script is included that detects which .net version is installed and copies the correct application and requirements to the system and launches the config.exe application.

Copying the files – The first step in setting up the Data Collector is to simply copy the programs to a folder on your computer. Note that we provide 2 versions depending on the .net framework that is installed (either .net 3.5 or .net 4 and above – if both are installed use the .net 4 or above version). The recommended location would be something similar to “C:\ProgramData\AppAdminTools\DataCollector\” – This folder should not use more than 150MB of space.

Launch the Config.exe Program – Once the files are copied to the system, simply launch the Config.exe program.

The configuration utility allows you to adjust the how the client works (all options are written to an xml file and can also be edited manually). The config tool also allows you to register the Scheduled Tasks to collect the data.

SPASS Data Collection Config Utilit

ServerName – allows you to override what the actual server name is and SPASS will use that

Output Directory – the Share Location where all the performance data will be stored (see Setting Up the Data Store for more info)

Summary Output Directory – the Share Location where the “current stats” are stored (see Setting Up the Data Store for more info)

Perflog Location – This is the location where the perfmon blg files are located at. If it is set to auto, the hourly data collector program will control perfmon and manage the data to ensure minimal drive space usage (usually only 80-100MB is needed).

Connection Stats – This allows the graphing of the data collected by the “countconnections” utility we provide to keep track of the connected users for applications that don’t keep track of that statistic

Provide Stats – This allows you to determine what stats are collected. By default it will automatically detect how many CPUs / Disks are on the server and provide stats for all cores/disks. Also if you do not need IIS or SQL stats, simply uncheck the boxes

Email Alerts – This enables the sending of emails when counters cross a threshold. Note that the “ProcessCurrent” application sends out the alerts, by default this program will run every 15 minutes

Register Scheduled Tasks

To start the Data Collection you must click the “Register Scheduled Tasks” button. This will add the 3 scheduled tasks required for data collection:

The Scheduled Tasks that perform the Data Collection

The CurrentSummaryCollection collects data for the last 3-4 hours and will send out the alert emails, which means that if an issue is lengthy you will get 4 emails an hour.

The HourlyCollection manages the Perfmon collection if it is set to Auto (adds, starts/stops the Data collector) as well as provides the hourly stats.

The DailyCollection collects data for the previous day.

Again, we decided to use standard Windows functions as opposed to Windows services to ensure it is easy to maintain as well as easy to roll it into your software management app such as SCCM or automation app such as Orchestrator. Most config settings will automatically adjust themselves for the server’s configuration (such as servername, multi-cpu, multi-disk, etc.)

If you do push it out, you really just need to pre-populate the Data Store options prior to pushing it out. Also note it is recommended to keep the Scheduled Task names and folder location the same to ensure someone doesn’t register multiple scheduled tasks (the config application checks these and will disable the Register Tasks button if all are present).

Collecting Data Across the Network

The network collection executables allow you to have a central server that will process the data for your network. This allows you to process operating systems that are otherwise not supported (i.e. Windows Server ® 2003 R2 and before) or if you would rather capture the data across the network instead of on the servers directly.

To collect data across the network, we provide 4 separate “nodes” that can be ran on a single server increasing the number of servers a single server can handle. To setup a Network Data Collector, first you copy the required files to the server that will collect the data (i.e. C:\ProgramData\AppAdminTools\NetworkCollector\). Then for each “Node” that you want to implement, run the “AppConfig.exe” application, adjust the settings accordingly and Register the Scheduled Tasks.

SPASS Network Collector Configuration

Output Directory – the Share Location where all the performance data will be stored (see Setting Up the Data Store for more info)

Summary Output Directory – the Share Location where the “current stats” are stored (see Setting Up the Data Store for more info)

SMTP Server – The SMTP server that you want to use to send out Email Alerts

Email From Address – The email address that the Alert Emails that will be sent from

Adjust the Network Collector Run As Account

Once you register the Scheduled Tasks, you will probably need to adjust the “Run As” account to something different than the local System account. The account you choose must have access to read the BLG Files, perform WMI queries and pull the System, Application and Security Logs from the remote server.

Adding Perfmon Data Collector on the remote servers

To be able to collect data from remote servers, the remote servers must be configured to collect performance data to Perfmon BLG files. Sample Templates are included that will capture quite a few Performance Counters to install these follow these directions:

Importing Perflog Templates on Windows® 2003 Servers

Importing Perflog Templates on Windows® 2008 Servers

Alternatively, if you just want to capture the counters that SPASS will process, setup a Data Collector that includes these counters:

\Memory\Pool Nonpaged Bytes
\Memory\Available Bytes
\Memory\Cache Bytes
\Memory\Page Reads/sec
\Memory\Page Writes/sec
\Network Interface(*)\Bytes Received/sec
\Network Interface(*)\Bytes Sent/sec
\Paging File(_Total)\% Usage
\Process(_Total)\Working Set
\Processor(*)\% Processor Time
\Processor(_Total)\% Privileged Time
\Processor(_Total)\% User Time
\SQLServer:Access Methods\Page Splits/sec
\SQLServer:Buffer Manager\Buffer cache hit ratio
\SQLServer:Buffer Manager\Lazy writes/sec
\SQLServer:Databases(_Total)\Log Flushes/sec
\SQLServer:General Statistics\Processes blocked
\SQLServer:General Statistics\User Connections
\SQLServer:Latches\Average Latch Wait Time (ms)
\SQLServer:Locks(_Total)\Average Wait Time (ms)
\SQLServer:Locks(_Total)\Number of Deadlocks/sec
\SQLServer:Memory Manager\SQL Cache Memory (KB)
\SQLServer:Memory Manager\Target Server Memory (KB)
\SQLServer:Memory Manager\Total Server Memory (KB)
\SQLServer:SQL Statistics\Batch Requests/sec
\SQLServer:SQL Statistics\SQL Compilations/sec
\SQLServer:SQL Statistics\SQL Re-Compilations/sec
\System\Processor Queue Length
\Web Service(_Total)\Current Anonymous Users
\Web Service(_Total)\Current Connections
\Web Service(_Total)\Current NonAnonymous Users
\Web Service(_Total)\Get Requests/sec
\Web Service(_Total)\Post Requests/sec
\Web Service(_Total)\Put Requests/sec

Adding the Remote Server to the Network Data Collector

To add the remote server to the network data collector you need to add the server information to the xml files under the ServerConfigs directory. You can simply add the server to an existing XML file (or create a new one). The required settings to be added are below, each server you want to process needs to have a separate <computer> entry:

   <server_description>Server Description</server_description>
   <connections_title>WebServer Connections </connections_title>

Descriptions for the above settings:

servername – This is the name of the remote system that you want to collect data for.

server_description – This is the description that is used within the web application. It is also used with the applications to show the current computer that is being processed in the terminal.

enabled – This provides an easy way to disable a system without deleting it’s configuration setting. This needs to be set to “true” for the applications to process data for the system.

perflog_file_location – This is the location of the Performance Montior BLG files for the system. To process remote systems, this needs to be the network location of the “BLG” files, i.e. \\servername\c$\perflogs\

multicpustats – If the system has multiple processors/cores, set this to true to provide detailed stats for each cpu/core

multidiskstats – If the system has multiple disk drives, set this to true to provide detailed stats for each disk drive

do_iis_stats – Set this to true to process stats from the collected IIS/Webservice Performance Counters

do_sql_stats – Set this to true to process stats from the collected SQL Performance Counters

do_connections – Set this to true if the system has a “Connections” file that was generated from our custom user connections application and you want to process data from it.

connections_title – Set this to what you want the resultant “Connections” Graph’s Title to be.

connections_path – Set this to the location of the “Connections” Application data files. To process data for remote systems, this needs to be the network location of the files, i.e. “\\servername\c$\connections\”)

provide_alerts – Set this to true if you want the “Current” application to send out email alerts

email_addresses – This should be a comma separated list of email addresses you want the alerts to be sent to

cpu_threshold – Set this to the percentage that you want the CPU Usage alert to be triggered on. For example, if set to 95 and the average of the Total CPU counters over the last 60 collections are above 95, an email alert will be sent out. This is generated from data in the BLG files

mem_threshold – Set this to the percentage of Used Memory that you want the alert to be triggered on. For example, if set to 95 and the percentage of “available memory” drops below 5%, an email alert will be sent out. This is generated from WMI data

nonpaged_pool_threshold – Set this to the percentage of NonPaged Pool memory usage that you want the alert to be triggered on. For example, if set to 50 and the system’s Non-Paged Pool memory is taking over 50% of the total system memory, an email alert will be sent out. This was added during testing since a couple of customers were experiencing Kernel memory leaks. This is generated from WMI data

pagefile_threshold – Set this to the percent used of the Pagefile. For example, if you set this to 95 and the pagefile usage goes above 95 percent, an email alert will be sent out. Note that this is only useful if you manually set the pagefile size on your systems. This is generated from WMI data

stopped_service_threshold – Set this to the number of “Automatic” services that don’t run all the time on the system, any number above the threshold will trigger an email alert. Note that the Web Frontend shows the number of “Stopped Services” that you can reference for this.

security_threshold – Set this to the number of Security Failures the Server detected. This will be over a 3-4 hour period and is useful to detect security breaches

diskspace_alert – Set this to the percent diskspace used for each disk in the system. For example, if you set this to 95 and the system’s C: drive only has 2% free space, an email alert will be sent out. This is generated from WMI data

Setting Up the SPASS Website

Once you have all the Data Collectors running on the servers you want to monitor, you need to setup the SPASS web sites to display and process the collected data. SPASS comprises of 2 different web frontends:

The “Single-Site” WebApp for each “Application Group”, which is a group of servers you want displayed together, an example would be a SharePoint Farm. This site is

The other is the “Multi-Site” WebApp, which acts like a “Landing Site” that lists all the Application Group Sites that you have configured as well as has the stats of all the servers that data is being collected for.

Web Site Requirements

To be able to host the SPASS web sites on your servers, you must host them on an IIS Server that has .net 4 installed and .net 4 has to be integrated with IIS (if you can’t set an App Pool to use a .net 4 app pool you need to install / register .net 4 with IIS). It is preferred to host the sites on IIS version 7 or above.

A “Virtual Directory” that points to the collected performance data is also required.

Copying the SPASS Web Files to the Webserver

The first step in setting up the website is to copy the SPASS_”Single Site” Web folder to an IIS webserver, by default you would copy the folder to a subfolder of “C:\inetpub\wwwroot”, such as “C:\inetpub\wwwroot\spass”. If you have quite a few computers that you are monitoring, you can create as many copies of “SPASS_Web” as you need to separate out the servers for better organization. Multiple “Application Group” sites are also recommended if you are planning on utilizing the “Multi-Site Web Frontend”.

As an example, one of our larger customers’ home office has 3 web server licenses (2 for production servers setup in a fault tolerant IIS cluster and 1 for non-prod servers hosted in another data center). The folder structure of the production site is below, 11 “SPASS Application sites” provide stats for 250+ Production Servers and the “SPASS MultiSite” provides the Landing site that brings all the Application Sites together.

SPASS Web Sites Folder Hierarchy

Note that each of the above folders are set as Applications within IIS using the same .net 4 AppPool. Also, the “Multi-Site” files are not listed in the above screen-shot to avoid confusion. If you are using the Multi-Site Web Frontend, you would copy those files to the main directory (in the example above it would be “C:\inetpub\wwwroot\AppAdmin”), then set that directory as an Application using a .net 4 App Pool.

Setting Up the .net 4 Application Pool

Before converting the copied SPASS_Web folder(s) as applications within IIS, it is recommended to create a new .net 4 AppPool that the site(s) will use. This process is straightforward:

  1. Open IIS Manager
  2. Click on Application Pools in the left pane
  3. Click on “Add Application Pool…” under the actions menu
  4. Fill in the desired name and ensure you select .net 4 as the .net version
  5. Ensure the AppPool is set to start immediately and hit OK

Creating a new Application Pool for the SPASS sites to use

If you are unable to select .net 4 for the .net version you will need to either install .net 4 or ensure it is registered within IIS (i.e. from elevated cmd, “cd C:\Windows\Microsoft.Net\Framework64\v4.0.30319” and run “aspnet_regiis.exe -i”, then perform an “iisreset”).

Creating a Virtual Directory for the Data Store

To allow IIS to display the pre-generated performance and disk graphs you have to setup what is called a “Virtual Directory” that points to the folder that the Data Collection Applications write to.

Adding a Virtual Directory within IIS

To create the Virtual Directory, (within the IIS Manager) simply browse to the website that will host SPASS_Web and right click on it and select “Add Virtual Directory”. This will launch the Add Virtual Directory Wizard.

The Add Virtual Directory Wizard

Using this wizard, you will specify an “Alias” (which “directory” on the web server points to the files) and the Physical Path. The Physical Path will usually be a local directory, such as “D:\Perfstats” or it could be a network share if you stored the actual files on a remote server, such as “\\servername\sharename”. The “Connect as…” button allows you to specify a user account that IIS will use to access the files. For files stored on the local server this usually is not needed, just be sure that the default IIS user has access to view the folders/files (or the user that the AppPool runs as if you adjusted advanced AppPool features).

Adjusting the Web.Config settings

To configure each SPASS “Application Site” and the SPASS Multi / Landing Site, you have to edit each site’s web.config file and specify the locations of the Data Store, the Virtual Directory you created as well as the Site’s Title that you want to use. An example web.config from a single App/Group site is shown below:

      <setting name="SummaryRoot_Directory" serializeAs="String">
      <setting name="ImageRootLocation" serializeAs="String">
      <setting name="HourlyRootDir" serializeAs="String">
      <setting name="HourlyImageLocation" serializeAs="String">
      <setting name="SiteTitle" serializeAs="String">
        <value>SPASS - Server Performance and Sizing Site</value>

SummaryRoot_Directory – This setting should point to the location that houses the “current” statistics. This location needs to be either a local path or a UNC share (i.e. not an IIS web location)

ImageRootLocation – This setting should describe the IIS web location of the “current” statistics. This is used to pass the image locations to the end user’s browsers for display. This should be the location from the root of the web site – i.e. “/perfstats/current” – where “perfstats” is the virtual directory that was setup earlier.

HourlyRootDir – This setting should point to the location that houses the “hourly” and “daily” statistics. This location needs to be either a local path or a UNC share (i.e. not an IIS web location)

HourlyImageLocation – This setting should describe the IIS web location of the “hourly” and “daily” statistics. This is used to pass the image locations to the end user’s browsers for display. This should be the location from the root of the web site – i.e. “/perfstats” – where “perfstats” is the virtual directory that was setup earlier.

SiteTitle – This setting is used to easily customize the Website’s Title that is displayed.

The Multi-Site’s web.config has the same settings and needs to be adjusted accordingly.

Converting Folders to Applications

Once the files are in place and each web.config file is updated for your environment, the next step is to convert the folder(s) to Web Applications.

Converting folders to Web Applications lets IIS know to process the site as an application using the specified AppPool. Note that on the web server, every AppPool is a w3wp.exe process. Each AppPool has it’s own running process.

The first step in converting the folder(s) to Web Applications is to locate the folder within IIS Manager and right click on it and select “Convert to Application”.

Converting a Folder to an Application

This will bring up the “Add Application” Wizard. Here you can specify the Application Pool that the Application will use. If you created a separate one for SPASS as described earlier, use that one, otherwise any .net version 4 AppPool will suffice.

The Add Application Wizard within IIS
Selecting the SPASS .net 4 App Pool

Once the folder(s) are converted to applications you can hit the Web Application within a Browser to ensure it comes up correctly. Initially for the “Single Application Group Sites” there will only be a few dummy servers listed, “ServerName” and “ServerName2”. The next section describes how to add servers to the AppGroup Web Application.

Adding Servers to the “Single Application Group” Web Apps

In order to add servers to the Single Application Groups Web Apps, you simply need to edit the servers.xml file located in the “ServerConfigs” directory of the App/Group Site’s folder, an example is below:

<?xml version='1.0'?>
      <server_description>Server Description</server_description>
      <server_description>Server Description</server_description>

Where each server you want added has it’s own “<computer>” section and has the “servername” and “server_description” options listed as shown below.

	<server_description>Basic Web Server</server_description>

Note that the ServerConfigs Directory should only have a single XML file in it. If additional files are present, only one will be processed.

Uninstalling SPASS

Since we created SPASS to be easy to deploy using your existing deployment tools, it is also extremely easy to un-install.

For the clients that you have deployed, you simply need to remove the actual Files from the system, delete the 3 scheduled tasks and delete the “AppAdminTools_SPASS” Data Collector from Perfmon if you configured the client automatically control the Perflog BLG files.

For the web server, you simply need to “remove the applications” within IIS Manager, then just delete the files. You can also delete the AppPool that you may have created for the Web Apps.

If you just removed a single client and you want to remove it from the Multi-Web Site, simply delete that server’s folder in the Data Store/Share that SPASS uses.

Basic Troubleshooting

If you are experiencing issues with SPASS, check these common issues and if you still require assistance, email

Inconsistent Data Collection on Busy Servers

If you are experiencing sporadic Data Collection failures on servers that utilize quite a bit of the CPU, such as every other hour graphs fail to generate, try to manually import a Perfmon Data Collector and point the data collector and point the apps to use those instead of automatically managing the Perflog BLG files.

This issue happens when our Data Collection applications finish before the Perflog Data Collector has a chance to fully stop, which prevents it from starting again.

Network Collectors failing to pull Application, System and Security Logs

If you are running the Network Data Collector remotely against a Windows 2003 Server, the Application, System and Security Logs will not be collected.

Running the Network Data Collector on a Windows Server 2008 machine will not pull the App, System and Security Logs from any Server. The Remote Data Collectors need to be ran on a Windows 2008R2 Server or higher for all the features to function correctly.

Web Site Throwing ASP Error

If the web site is throwing an error instead of displaying correctly, first ensure that you configured the Web App’s Folder as an Application and that it uses a .net 4 Application Pool.

If you are still having issues with IIS errors, check the application log and read the exact error that is occurring. Many ASP errors are related to malformed XML files and even though we sanitize the files, sometimes an odd character is written to those files. In this instance, email us an example of the file that the web site is having issues with and we will look at resolving the issue.

Alert Emails not sending out

If you suspect the alert emails are not sending out, first check the “current” logs to see if the program sent an email. If the log showed an email should have been sent and you did not receive it, you can do a rudimentary test to ensure the .net libraries are able to send an email through your mail server. To do this, adjust and run this PowerShell script that uses the same .net libraries that SPASS uses:

$PSEmailServer = ""
$mailtoaddress = ""
$fromaddress = ""

$messageheader = ".Net Test Email"
$messagebody = "This is a test email to ensure .net is sending out emails"
Send-MailMessage -From $fromaddress -To $mailtoaddress -Subject $messageheader -Body $messagebody

If the script is able to send an email and yet SPASS does not, another troubleshooting process would be to use a network capture utility, such as Wireshark and capture port 25 when the Current Data Collector sends an alert email. This should provide enough information for you to resolve the issue.

Useful Scripts

Below are some example scripts that may aid in deploying and maintaining SPASS, a copy of these scripts are located within the installation archive. These examples are unsupported and use at your own risk.

Delete BLG Files PowerShell Script

This PowerShell script allows you to delete Perflog BLG files that are older than a certain number of days, this should be ran daily as a scheduled task.

$file_query = "*.blg"
$CurrentDate = Get-Date
$deldate = 2
$file_location = "C:\perflogs\"
$files=get-childitem $file_location\$file_query | where-object {!($_.psiscontainer)}
if ($files.count -gt 0) {
  ForEach ($file in $files) {
    $file_modified = $file.LastWriteTime
    if ($file_modified -gt $CurrentDate.AddDays(-$deldate) ) { "$file Not old enough" }
    Else {
      Remove-Item $file

Install Client DOS BATch file

Here is an example DOS bat file that is a rudimentary installation script for the Local Data Collector. Simply copy the files to a network share (adjust as necessary and pre-populate a config.xml file), then run the script on a server that you want to add monitoring to.

It will copy the correct files for the server, optionally start the pre-req requirements if the server has only .net 3.5, then launch the config app, which you just need to click the schedule tasks button.

if not exist "C:\ProgramData\AppAdminTools" ( mkdir "C:\ProgramData\AppAdminTools")

Timeout 2

if exist "C:\Windows\Microsoft.NET\Framework\v4.0.30319" ( robocopy \\server\share\AppAdminTools\SPASS\DataCollector C:\ProgramData\AppAdminTools\DataCollector /E )

Timeout 2

if not exist "C:\ProgramData\AppAdmin\DataCollector" ( robocopy \\server\share\AppAdminTools\SPASS\DataCollector_3.5 C:\ProgramData\AppAdminTools\DataCollector_3.5 /E )

Timeout 2

if exist "C:\ProgramData\AppAdminTools\DataCollector_3.5" ( \\server\share\AppAdminTools\SPASS\dotnet35_reqs\MSChart.exe )

Timeout 2

if exist "C:\ProgramData\AppAdminTools\DataCollector" ( C:\ProgramData\AppAdminTools\DataCollector\config.exe )

Timeout 2

if exist "C:\ProgramData\AppAdminTools\DataCollector_3.5" ( C:\ProgramData\AppAdminTools\PerfDataCollector_3.5\config.exe )						

Delete Old Hourly PerfStats PowerShell Script

Here is an example of how to clear out old “hourly” performance stats that you may no longer need in order to clear up disk space. Removing hourly performance stats will prevent the “Hourly” and “Daily” Pages from generating, but the Weekly and Monthly stats will continue to function to provide you those graphs. This can be ran as a weekly or monthly scheduled task.

$perfstats_dir = "E:\Perfstats"
$deleteage = 90
$Now = Get-Date
$logfile = New-Item -type file -force "C:\ProgramData\AppAdminTools\perflogcleanup.txt"

$starttime = Get-Date
add-content $logfile "Job Starting - $Now"

$computers=get-childitem $perfstats_dir | where-object {($_.psiscontainer)}

ForEach ($computer in $computers) {
  echo $computer
  $currtime = Get-Date
  add-content $logfile "$currtime - $computer"
  $years = get-childitem $perfstats_dir\$computer | where-object {($_.psiscontainer)}
  ForEach ($year in $years) {
    $months = get-childitem $perfstats_dir\$computer\$year | where-object {($_.psiscontainer)}
    ForEach ($month in $months) {
      $days = get-childitem $perfstats_dir\$computer\$year\$month | where-object {($_.psiscontainer)}
      ForEach ($day in $days) {
        Remove-Variable hours
        $hours = get-childitem $perfstats_dir\$computer\$year\$month\$day | where-object {($_.psiscontainer)}
        if ($hours) {
          ForEach ($hour in $hours) {
            Remove-variable files
            $files=get-childitem $perfstats_dir\$computer\$year\$month\$day\$hour | where-object {!($_.psiscontainer)}
            if (!($files)) { Remove-Item $hour.fullname -Recurse }
              ForEach ($file in $files) {
                $file_modified = $file.LastWriteTime
                if ($file_modified -lt $Now.AddDays(-$deleteage) )
                  echo $file.fullname
                  Remove-Item $file.fullname

$endtime = Get-Date
$difftime = $endtime - $starttime
$totalminutes = $difftime.TotalMinutes

add-content $logfile "$endtime - Job Ended, it took $totalminutes minutes to complete"