mercredi 30 septembre 2015

How to design logging in MSTest test suite such that each test case writes to it's own log file

What I want to do

I want to include logging in my tests preferably using System.Diagnostics such that each test case writes to its own log file (let's say the format is {test-case-name}.log). The catch is that the logging statements will not be in the test methods themselves, but in a separate class hierarchy, which the different test methods will share. An example:

Test classes:

  1. BaseTestClass (abstract)
  2. DerivedTestClass_Firefox: BaseTestClass (has test methods like TestA_Firefox,TestB_Firefox)
  3. DerivedTestClass_Chrome: BaseTestClass (has test methods like TestA_Chrome, TestB_Chrome)

OperationClass (includes basic actions done by tests and has the logging statements): Let's say it has methods: OperationA, OperationB, etc. Now different test methods can share these operations. So if OperationA has Logger.LogInfo("Doing operation A"), then this message should go to the correct log file according to what test called it. The tests need to be able to run in parallel. I will be using MSTest if it matters.

My current solution

I made a static Logger class like below:

    public static class Logger
    {
        //stores tracesource objects against tracesource names (same as test method name)
        //entries are added in this by the [TestInitialize] method in BaseTest, as it has access to the calling test method name (from TestContext)
        private static Dictionary<string, TraceSource> TraceSourceMap = new Dictionary<string, TraceSource>();
        //automatically determines which log file the message should go to
        //from the stack trace and test method naming conventions
        private static string GetCallingTestName()
        {
            StackTrace stackTrace = new StackTrace(); 
            StackFrame[] stackFrames = stackTrace.GetFrames();

            foreach (StackFrame stackFrame in stackFrames)
            {
                if (stackFrame.GetMethod().Name.ToLower().EndsWith("_firefox") ||
                    stackFrame.GetMethod().Name.ToLower().EndsWith("_chrome") ||
                    stackFrame.GetMethod().Name.ToLower().EndsWith("_ie"))
                {
                    return stackFrame.GetMethod().Name;
                }
            }
            return null;

        }
        private static TraceSource GetTraceSourceObject()
        {
            string traceSourceName = GetCallingTestName();
            TraceSource ts = TraceSourceMap[traceSourceName];
            return ts;
        }
        public static void LogInfo(string logMessage)
        {                      
            TraceSource ts = GetTraceSourceObject();
            ts.TraceInformation(String.Format("[{0}] {1}\n", DateTime.Now.TimeOfDay, logMessage));

        }
   }

The [TestInitialize] method in BaseTestClass sets up TraceSource with the name as the calling test method name, adds the listener (a log file with same name as test name) and then adds this object in Logger class's dictionary.

The issue is that this design completely depends on test naming. The tests must have _firefox, _chrome, etc. in the end, and none of the other methods can end like this, because then it'll be impossible to tell through stack trace what is a test method or what is an operation (unless I impose further pattern in test method naming). On the other hand, this is very less code and will work perfectly even if I run tests in parallel.

Problem/ Question

I want to know in what other way I can achieve the same result and if this design is actually that bad. One idea might be to create a logger object in [TestInitialize] of BaseTestClass (recall that before this I don't know what the logging file will be as it depends on test name, so can't do this at constructor time) and then somehow pass this logger to OperationClass. But then there is only one instance of logger object for the whole test class and this would be a problem if I want to run tests in parallel right? (As each test method logs in a different place, so each test method needs its own "version" of logger object. Even for parallely running tests I guess the test class is instantiated only once?)

Aucun commentaire:

Enregistrer un commentaire