Peek into your network traffic
Perhaps the easiest way to determinenr if any nefarious behavior is taking place on your SQL Server is to see how it's talking on the network. If you have a network analyzer you're comfortable with, you can start rooting out what's going on in just a minute or two. You can load your analyzer on the SQL Server itself or have it connected elsewhere to a span or mirror port on your Ethernet switch.
My favorite network analyzer, EtherPeek, can be used like most other analyzers to capture packets going to or from your SQL Server. As shown in the figure below, some traffic running over TCP port 12345 (a common NetBus Trojan port) is discovered.
EtherPeek can easily capture network traffic highlighting Trojan behavior – in this case capturing NetBus traffic
You can actually create your own network analyzer triggers and filters if you know what to look for. A good listing of common Trojans and their associated ports can be found here. This method of rooting out malicious traffic isn't foolproof since port numbers can often be changed, but it serves as a good starting point.
You can run Ether Peek in 'monitor' mode to let it glean a bird's eye view of what's taking place on the network – without having to capture packets. You can view which protocols are in use as well as look for heavy traffic, odd hosts communicating, and other network trends to/from your SQL Server system. This is demonstrated in the following screenshot.
EtherPeek's monitor mode can highlight network trends such as Trojan communications you wouldn't otherwise know about

Test for a Trojan horse on your SQL Server

Home: Introduction
Step 1: Scan your SQL Server for malware
Step 2: Look in the memory
Step 3: Look at open ports
Step 4: Peek into your network traffic
Step 5: Approach with a malicious mindset