Crash when using third-party obfuscated dll

Answered

Gilles Ronsin

Created February 23, 2017 10:08

I am managing a complete CI solution for company's product.

For each C# product the complete string of tasks is cleaning/building/fxcop/stylecop/unit tests/code coverage/signature/obfuscation if needed/release candidate/multi platform verifications/release

We are using old NCover3 and Nunit 2.5.9 but try to migrate to dotCover/Nunit3.5.0

During work out all spends nicely.. until I set the obfuscation at the head of production.

Ok. I can understand than an obfuscated library is not usable to be covered. The solution is to have 2 sets of libraries: with and without obfuscation. One for dev, the other for delivery.

But, when the C# project, once strong named, references an obfuscated library (considered here as third-party library), the dotcover crash at third test with weird messages:

[10:15:42][Exec] An existing connection was forcibly closed by the remote host
[10:15:42][Exec]
[10:15:42][Exec] Server stack trace:
[10:15:42][Exec] at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.SocketStream.Read(Byte[] buffer, Int32 offset, Int32 size)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.SocketHandler.ReadFromSocket(Byte[] buffer, Int32 offset, Int32 count)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.SocketHandler.BufferMoreData(Byte[] dataBuffer)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.SocketHandler.Read(Byte[] buffer, Int32 offset, Int32 count)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.SocketHandler.ReadAndMatchFourBytes(Byte[] buffer)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.Tcp.TcpSocketHandler.ReadAndMatchPreamble()
[10:15:42][Exec] at System.Runtime.Remoting.Channels.Tcp.TcpSocketHandler.ReadVersionAndOperation(UInt16& operation)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.Tcp.TcpClientSocketHandler.ReadHeaders()
[10:15:42][Exec] at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
[10:15:42][Exec] at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)
[10:15:42][Exec]
[10:15:42][Exec] Exception rethrown at [0]:
[10:15:42][Exec] at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
[10:15:42][Exec] at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
[10:15:42][Exec] at NUnit.Engine.ITestEngineRunner.Run(ITestEventListener listener, TestFilter filter)
[10:15:42][Exec] at NUnit.Engine.Runners.ProcessRunner.RunTests(ITestEventListener listener, TestFilter filter)
[10:15:42][Exec]
[10:15:42][Exec] Test Run Summary
[10:15:42][Exec] Overall result: Failed
[10:15:42][Exec] Test Count: 0, Passed: 0, Failed: 0, Inconclusive: 0, Skipped: 0
[10:15:42][Exec] Start time: 2017-02-23 09:15:41Z
[10:15:42][Exec] End time: 2017-02-23 09:15:44Z
[10:15:42][Exec] Duration: 3.799 seconds
[10:15:42][Exec]
[10:15:42][Exec] Results (nunit3) saved as TestResult.xml
[10:15:42][Exec]
[10:15:42][Exec] No connection could be made because the target machine actively refused it 127.0.0.1:60342
[10:15:42][Exec] Failed to stop the remote agent. No connection could be made because the target machine actively refused it 127.0.0.1:60342
[10:15:44][Exec] [JetBrains dotCover] Coverage session finished [23/02/2017 10:15:50]
[10:15:44][Exec] [JetBrains dotCover] Analysed application exited with code '-100'

Of course, the tested code does not related to TCP, just call a method of third party library (which does not related to TCP too)

No issue if strong naming not enabled.

No issue using NCover.

I was supposed it was something wrong with pdb....

9 comments

Alexander Mikhailov

Created February 24, 2017 13:20

Hello Gilles!

Looks like NUnit agent process has crashed and NUnit was not able to communicate with it.

To investigate this further I would like to ask you to:

- Use the latest NUnit version (I think it is 3.6)

- Use --inprocess option for NUnit console (if possible)

- Send us dotCover log files.

To enable dotCover logs add "/logfile" switch to the end of dotCover console command line.

Logs will be saved to %TEMP%\JetLogs. Please clear this folder before running any tests. And send us all logs that will appear there after.

Gilles Ronsin

Created February 27, 2017 08:00

Hi Alexander,

Thank you for answer:

here are logs files: http://dl.free.fr/wfb7Zo6hB

Alexander Mikhailov

Created February 27, 2017 09:34

Gilles,

thanks for logs. Could you also show us console output for this run?

Gilles Ronsin

Created February 27, 2017 09:47

I suppose than the teamcity log, is the same output than console:

[08:36:25]
[UnitTesting]
========
TestAssemblyList:
D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Build.CI\Targets\..\..\Outputs\NUnit\Files\TSF.CBS.GenericDriver.Tests.dll
[08:36:25][UnitTesting] XmlPeek
[08:36:25]
[UnitTesting] #####

D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\Global.Build.CI\LocalTools\NUnit\3.6.0\nunit3-console.exe
D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Build.CI\Targets\..\..\Outputs\NUnit\Files
D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Build.CI\Targets\..\..\Outputs\NUnit\Files\TSF.CBS.GenericDriver.Tests.dll --inprocess --teamcity
D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Build.CI\Targets\..\..\Outputs\NUnit\dotCoverReport.dcvr

TSF.CBS.GenericDriver.Framework
TSF.CBS.GenericDriver.Public
TSF.CBS.GenericDriver.Tests.TestDriver.Implementation
TSF.CBS.GenericDriver.Tests.TestDriver
TSF.CBS.GenericDriver

*TestDriver*

System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute
System.CodeDom.Compiler.GeneratedCodeAttribute

[08:36:24][Step 5/6] Targets were not defined in the build configuration.
[08:36:25][UnitTesting] Exec (14s)
[08:36:25][Exec] C:\BuildAgent\Tools\dotCover\dotCover.exe cover D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Build.CI\Targets\..\..\Outputs\NUnit\DotCoverConfig.xml /logfile
[08:36:25][Exec] JetBrains dotCover Console Runner 2016.3.2. Build 107.0.20170126.121718
[08:36:25][Exec] Copyright (c) 2009-2017 JetBrains s.r.o. All rights reserved.
[08:36:31][Exec] [JetBrains dotCover] Coverage session started [27/02/2017 08:36:36]
[08:36:32][Exec] NUnit Console Runner 3.6.0
[08:36:32][Exec] Copyright (C) 2017 Charlie Poole
[08:36:32][Exec]
[08:36:33][Exec] Runtime Environment
[08:36:33][Exec] OS Version: Microsoft Windows NT 6.0.6002 Service Pack 2
[08:36:33][Exec] CLR Version: 4.0.30319.42000
[08:36:33][Exec]
[08:36:33][Exec] Test Files
[08:36:33][Exec] D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Outputs\NUnit\Files\TSF.CBS.GenericDriver.Tests.dll
[08:36:33][Exec]
[08:36:34][Step 5/6] TSF.CBS.GenericDriver.Tests.dll (running for 9s)
[08:36:34][TSF.CBS.GenericDriver.Tests.dll] TSF.CBS.GenericDriver.Tests.AssemblyLoaderTests.TestLoad
[08:36:35][TSF.CBS.GenericDriver.Tests.dll] TSF.CBS.GenericDriver.Tests.AssemblyLoaderTests.TestLoadException (running for 8s)
[08:36:39][Exec] [JetBrains dotCover] Coverage session finished [27/02/2017 08:36:45]
[08:36:39][Exec] [JetBrains dotCover] Analysed application exited with code '-2146232797'
[08:36:39][Exec] [JetBrains dotCover] Coverage session finished but no snapshots were created.
[08:36:39][Exec] D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Build.CI\Targets\UnitTesting.targets(221, 9): error MSB3073: The command "C:\BuildAgent\Tools\dotCover\dotCover.exe cover D:\Work\RE_Dotcover_Business_Drivers_x32_RS_Client\dev\GenericDriver\Build\Build.CI\Targets\..\..\Outputs\NUnit\DotCoverConfig.xml /logfile" exited with code -2.
[08:36:39][Step 5/6] Process exited with code 1

Alexander Mikhailov

Created February 27, 2017 13:05

Gilles,

I was not able to reproduce this issue. Maybe you can create a minimal reproducing sample for us?

Gilles Ronsin

Created February 27, 2017 13:15

Hi Alexander,

I will try to have some time to build a sample. Just have 2 or 3 hot spots in progress

Just to know, one of referenced dll is built, obfuscated by confuserEx (with 'anti-debug' option), strong named and finaly signed.

Thank you spending time.

Gilles Ronsin

Created March 01, 2017 09:28

Hi Alexander,

Here are steps producing crash.. (not the same error but maybe it could help)

1) an assembly with a marvelous method. It will be my "external library" -> ObfuscatedSample.dll

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ObfuscatedSample
{
/// <summary>
/// The Class1.
/// </summary>
public class StringWork
{
public static string GetString(string source)
{
return "The sended string was " + source;
}
}
}

2) my code referencing my library UsingSample.dll

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace UsingObfuscatedSample
{
public class UsingSample
{
public static string Transform()
{
return ObfuscatedSample.StringWork.GetString("test");
}
}
}

3) My Unit Test class

using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using NUnit.Framework;

namespace UsingTests
{
[TestFixture]
public class UsingTests
{
[Test]
public void UsingTest()
{
Assert.IsNotEmpty(UsingObfuscatedSample.UsingSample.Transform());
}
}
}

4) build on CI, unit test, all is ok

5) releasing, strong naming, obfuscation, signature, now we will using an ObfuscatedSample.dll completely authentified and obfuscated

6) build a new client project Consumption.dll but references previous obfuscated dll (not the built)

namespace Consumption
{
public class UsingSample
{
public static string Transform()
{
return ObfuscatedSample.StringWork.GetString("test");
}
}
}

7) Unit test:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using NUnit.Framework;
using Consumption;

namespace ConsumptionTests
{
[TestFixture]
public class UsingTests
{
[Test]
public void UsingTest()
{
Assert.IsNotEmpty(UsingSample.Transform());
}
}
}

8) build on CI run unit test and... crash

[10:10:16][UnitTesting] Exec (8s)
[10:10:16][Exec] C:\BuildAgent\Tools\dotCover\dotCover.exe cover D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\..\..\Outputs\NUnit\DotCoverConfig.xml /logfile
[10:10:17][Exec] JetBrains dotCover Console Runner 2016.3.2. Build 107.0.20170126.121718
[10:10:17][Exec] Copyright (c) 2009-2017 JetBrains s.r.o. All rights reserved.
[10:10:18][Exec] [JetBrains dotCover] Coverage session started [01/03/2017 10:10:24]
[10:10:19][Exec] NUnit Console Runner 3.6.0
[10:10:19][Exec] Copyright (C) 2017 Charlie Poole
[10:10:19][Exec]
[10:10:20][Exec] Runtime Environment
[10:10:20][Exec] OS Version: Microsoft Windows NT 6.0.6002 Service Pack 2
[10:10:20][Exec] CLR Version: 4.0.30319.42000
[10:10:20][Exec]
[10:10:20][Exec] Test Files
[10:10:20][Exec] D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Outputs\NUnit\Files\ConsumptionTests.dll
[10:10:20][Exec]
[10:10:20][Step 5/6] ConsumptionTests.dll (running for 8s)
[10:10:20][ConsumptionTests.dll] ConsumptionTests.UsingTests.UsingTest (running for 8s)
[10:10:25][Exec] [JetBrains dotCover] Coverage session finished [01/03/2017 10:10:31]
[10:10:25][Exec] [JetBrains dotCover] Analysed application exited with code '-2146232797'
[10:10:25][Exec] [JetBrains dotCover] Coverage session finished but no snapshots were created.
[10:10:25][Exec] D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\UnitTesting.targets(215, 3): error MSB3073: The command "C:\BuildAgent\Tools\dotCover\dotCover.exe cover D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\..\..\Outputs\NUnit\DotCoverConfig.xml /logfile" exited with code -2.
[10:10:25][Step 5/6] Process exited with code 1
[10:10:25][Step 5/6] MSBuild output
[10:10:25][MSBuild output] Copyright (c) 2009-2017 JetBrains s.r.o. All rights reserved.
[10:10:25][MSBuild output] [JetBrains dotCover] Coverage session started [01/03/2017 10:10:24]
[10:10:25][MSBuild output] NUnit Console Runner 3.6.0
[10:10:25][MSBuild output] Copyright (C) 2017 Charlie Poole
[10:10:25][MSBuild output]
[10:10:25][MSBuild output] Runtime Environment
[10:10:25][MSBuild output] OS Version: Microsoft Windows NT 6.0.6002 Service Pack 2
[10:10:25][MSBuild output] CLR Version: 4.0.30319.42000
[10:10:25][MSBuild output]
[10:10:25][MSBuild output] Test Files
[10:10:25][MSBuild output] D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Outputs\NUnit\Files\ConsumptionTests.dll
[10:10:25][MSBuild output]
[10:10:25][MSBuild output]
[10:10:25][MSBuild output]
[10:10:25][MSBuild output] [JetBrains dotCover] Coverage session finished [01/03/2017 10:10:31]
[10:10:25][MSBuild output]
[10:10:31][MSBuild output] ##teamcity[message text='|[JetBrains dotCover|] Analysed application exited with code |'-2146232797|'' status='WARNING']
[10:10:25][MSBuild output]
[10:10:31][MSBuild output] ##teamcity[message text='|[JetBrains dotCover|] Coverage session finished but no snapshots were created. ' status='ERROR']
[10:10:25][MSBuild output] D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\UnitTesting.targets(215,3): error MSB3073: The command "C:\BuildAgent\Tools\dotCover\dotCover.exe cover D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\..\..\Outputs\NUnit\DotCoverConfig.xml /logfile" exited with code -2. [D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\UnitTesting.targets.teamcity]
[10:10:25][MSBuild output] Done Building Project "D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\UnitTesting.targets.teamcity" (UnitTesting target(s)) -- FAILED.
[10:10:25][MSBuild output]
[10:10:25][MSBuild output] Build FAILED.
[10:10:25][MSBuild output]
[10:10:25][MSBuild output] "D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\UnitTesting.targets.teamcity" (UnitTesting target) (1) ->
[10:10:25][MSBuild output] (UnitTesting target) ->
[10:10:25][MSBuild output] D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\UnitTesting.targets(215,3): error MSB3073: The command "C:\BuildAgent\Tools\dotCover\dotCover.exe cover D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\..\..\Outputs\NUnit\DotCoverConfig.xml /logfile" exited with code -2. [D:\Work\RE_Dotcover_2_Business_Drivers_x32_RS_Client\dev\ObfuscationConsumption\Build\Build.CI\Targets\UnitTesting.targets.teamcity]
[10:10:25][MSBuild output]
[10:10:25][MSBuild output] 0 Warning(s)
[10:10:25][MSBuild output] 1 Error(s)
[10:10:25][MSBuild output]
[10:10:25][MSBuild output] Time Elapsed 00:00:08.62
[10:10:25][ConsumptionTests.UsingTests.UsingTest] ConsumptionTests.dll (running for 3s)

Gilles Ronsin

Created March 01, 2017 09:29

NB: no logfile has been written

Alexander Mikhailov

Created March 01, 2017 10:46

Hello, Gilles!

thank you for a sample. Testing with ConfuserEx revealed lots of incompatibilities with dotCover. I would recommend you to exclude such "confused" assemblies from coverage by name via coverage filters. Coverage for obfuscated assemblies makes not much sense anyway.

I have also created an issue in our tracker. You are welcome to vote and track it's status.

Please sign in to leave a comment.