We do a lot of compiler work in RavenDB. Indexes are one such core example, where we take the C# language and beat both it and our heads against the wall until it agrees to do what we want it to.

A lot of that is happening using the excellent NRefactory library as well as the not so excellent CodeDOM API. Basically, we take a source string, convert it into something that can run, then compile it on the fly and execute it.

I decided to check the implications of using this using a very trivial benchmark:

private static void CompileCodeDome(int i)
{
    var src = @"
class Greeter
{
static void Greet()
{
System.Console.WriteLine(""Hello, World"" + " + i + @");
}
}";
    CodeDomProvider codeDomProvider = new CSharpCodeProvider();
    var compilerParameters = new CompilerParameters
    {
        OutputAssembly= "Greeter.dll",
        GenerateExecutable = false,
        GenerateInMemory = true,
        IncludeDebugInformation = false,
        ReferencedAssemblies =
        {
            typeof (object).Assembly.Location,
            typeof (Enumerable).Assembly.Location
        }
    };
    CompilerResults compileAssemblyFromSource = codeDomProvider.CompileAssemblyFromSource(compilerParameters, src);
    Assembly compiledAssembly = compileAssemblyFromSource.CompiledAssembly;
}

private static void CompileRoslyn(int i)
{
    var syntaxTree = CSharpSyntaxTree.ParseText(@"
class Greeter
{
static void Greet()
{
System.Console.WriteLine(""Hello, World"" + " +i +@");
}
}");

    var compilation = CSharpCompilation.Create("Greeter.dll",
        syntaxTrees: new[] {syntaxTree},
        references: new MetadataReference[]
        {
            new MetadataFileReference(typeof (object).Assembly.Location),
            new MetadataFileReference(typeof (Enumerable).Assembly.Location),
        });

    Assembly assembly;
    using (var file = new MemoryStream())
    {
        var result = compilation.Emit(file);
    }
}

I run it several times, and I got (run # on the X axis, milliseconds on the Y axis):

The very first Roslyn invocation is very costly. The next are pretty much nothing. Granted, this is a trivial example, but the CodeDOM (which invokes csc) is both much more consistent but much more expensive in general.