Saturday, 22 September 2012

WCF Serialization performance test

This is an application I wrote some time ago to try to determine the performance benefits of protobuf-net serialization when used with WCF.

(If running the program in Visual Studio be sure to use 'Run as Administrator' to start VS.)


You can configure various aspects like the number of client proxies and the number of calls to make to each proxy.

 WcfClient /?  
 Wcf Test Client...  
 Usage: 
 WcfClient [Verbose=False] [NamedPipes=True] [ServerAddress=localhost] [ProxyCount=1] [RunCount=1]  
 Possible arguments:  
 Help      Display Usage Help information.  
 Verbose   Turn on verbose logging to console.  
 NamedPipes   Use NamedPipes to talk to a local server process.  
 ServerAddress Servername / IP Address to use for remote server when not using NamedPipes  
 ProxyCount   The number of client proxies to run in parallel.  
 RunCount    The total number of test WCF calls to make across all proxies.  
   

Here is a basic run with just one proxy and one call.

 Wcf Test Client...  
 Running with ProxyCount=1, RunCount=1...  
   
 ---------- Run Basic Orders Tests ----------  
   
 Testing NetTcp Protobuf v2.4.0a ...  
 Setup test endpoint net.tcp://localhost:8754/WcfLoadTest/Proto  
 Time: 383ms, Bytes sent: 406, Bytes received: 585,507  
   
 Testing NetTcp ServiceStack v2.05 TypeSerializer ...  
 Setup test endpoint net.tcp://localhost:8755/WcfLoadTest/ServiceType  
 Time: 842ms, Bytes sent: 416, Bytes received: 1,080,166  
   
 Testing DataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8752/WcfLoadTest/Tuned  
 Time: 90ms, Bytes sent: 406, Bytes received: 585,507  
   
 Testing NetDataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8753/WcfLoadTest/Tuned  
 Time: 223ms, Bytes sent: 495, Bytes received: 415,674  
   
 Testing BasicHttp ...  
 Setup test endpoint http://localhost:8750/WcfLoadTest/Vanilla  
 Time: 159ms, Bytes sent: 365, Bytes received: 2,050,499  
   
 Testing Json WebHttp ...  
 Setup test endpoint http://localhost:8751/WcfLoadTest/Json  
 Time: 764ms, Bytes sent: 277, Bytes received: 1,207,660  
   
 ---------- Run Complex Graph Tests ----------  
   
 Testing NetTcp Protobuf v2.4.0a ...  
 Setup test endpoint net.tcp://localhost:8754/WcfLoadTest/Proto  
 Time: 234ms, Bytes sent: 416, Bytes received: 1,102,680  
   
 Testing DataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8752/WcfLoadTest/Tuned  
 Time: 138ms, Bytes sent: 416, Bytes received: 1,102,680  
   
 Testing NetDataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8753/WcfLoadTest/Tuned  
 Time: 130ms, Bytes sent: 501, Bytes received: 150,334  
   
 Testing BasicHttp ...  
 Setup test endpoint http://localhost:8750/WcfLoadTest/Vanilla  
 Time: 92ms, Bytes sent: 344, Bytes received: 1,662,004  
   
 Press any key to exit...  

Here is a run with 20 proxies and 80 service calls.

 Wcf Test Client...  
 Running with ProxyCount=20, RunCount=80...  
   
 ---------- Run Basic Orders Tests ----------  
   
 Testing NetTcp Protobuf v2.4.0a ...  
 Setup test endpoint net.tcp://localhost:8754/WcfLoadTest/Proto  
 Time: 3505ms, Bytes sent: 21,620, Bytes received: 46,814,400  
   
 Testing NetTcp ServiceStack v2.05 TypeSerializer ...  
 Setup test endpoint net.tcp://localhost:8755/WcfLoadTest/ServiceType  
 Time: 7010ms, Bytes sent: 22,360, Bytes received: 86,407,760  
   
 Testing DataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8752/WcfLoadTest/Tuned  
 Time: 3510ms, Bytes sent: 21,620, Bytes received: 46,814,400  
   
 Testing NetDataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8753/WcfLoadTest/Tuned  
 Time: 4992ms, Bytes sent: 24,240, Bytes received: 33,217,140  
   
 Testing BasicHttp ...  
 Setup test endpoint http://localhost:8750/WcfLoadTest/Vanilla  
 Time: 4923ms, Bytes sent: 27,376, Bytes received: 164,039,900  
   
 Testing Json WebHttp ...  
 Setup test endpoint http://localhost:8751/WcfLoadTest/Json  
 Time: 8056ms, Bytes sent: 20,360, Bytes received: 96,612,800  
   
 ---------- Run Complex Graph Tests ----------  
   
 Testing NetTcp Protobuf v2.4.0a ...  
 Setup test endpoint net.tcp://localhost:8754/WcfLoadTest/Proto  
 Time: 4002ms, Bytes sent: 21,820, Bytes received: 88,195,440  
   
 Testing DataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8752/WcfLoadTest/Tuned  
 Time: 3577ms, Bytes sent: 21,820, Bytes received: 88,195,440  
   
 Testing NetDataContract NetTcp ...  
 Setup test endpoint net.tcp://localhost:8753/WcfLoadTest/Tuned  
 Time: 2931ms, Bytes sent: 24,360, Bytes received: 11,999,420  
   
 Testing BasicHttp ...  
 Setup test endpoint http://localhost:8750/WcfLoadTest/Vanilla  
 Time: 2249ms, Bytes sent: 27,520, Bytes received: 132,960,300  
   
 Press any key to exit...  
   

There are two types of serialization tested basic flat non cyclic data and cyclic graphs. It can be seen from these results that in real world scenarios there is very little advantage to using Protobuf over the standard Microsoft serializers.

Some people have reported an improvement of 1 millisecond per call with large basic payloads using protobuf v1 (without graph support), however you would need to carefully consider your system to determine if this will be suitable in your system, for most people the MS serializers should be more suitable.

The actual test is rather more complicated than many similar tests, this is because it tries to more closely mimic a real world scenario. There are separate processes for the client and the server and the client passes the binding to the server using a management WCF service. The server (WCF host) will create and tear down services as requested by the management service. In this manner the client can automate and contol the server in order to control the number of endpoints and the binding types.

One issue with this approach is not all WCF bindings are serializable, however most are and I believe the test covers enough of the standard bindings to still be useful.

You can get the code here.

No comments:

Post a Comment