C# 7 Additions – Pattern Matching

C# 7 has started to introduce Pattern Matching. This is a concept found in functional programming, and although it isn’t fully implemented compared to F#, it is a step in that direction. Microsoft has announced they intend on expanding it in future releases.

Constant Patterns

The is keyword has been expanded to allow all constants on the right side of the operator instead of just a type. Previously, C#’s only valid syntax was similar to:

Now it is possible to compare a variable to anything which is a constant: null, a value, etc.

Behind the scenes, the is statement is converted to calling the Equals function in IL code. The following two functions produce roughly the same code (they call different overloads of the Equals function).

CheckIsNull

CheckEqualsNull

This can also be combined with other features allowing variable assignment through the is operator.

In Visual Studio Preview 4, the scoping rules surrounding variables assigned in this manner are more restrictive than in the final version. Right now, they can only be used within the scope of the conditional statement.

Switch Statements

The new pattern matching extensions have also extended and changed the use of case statements. Patterns can now be used in switch statements.

Like in previous versions, the default statement will always be evaluated last, but the location of the other case statements now matter.

In this example, case int n will never evaluate, because the statement above it will always be true. Fortunately, the C# compiler will evaluate this, determine that it can’t be reached and raise a compiler error.

The variables declared in patterns behave differently than others. Each variable in a pattern can have the same name without running into a collision with other statements. Just as before, in order to declare a variable of the same name inside the case statement, you must still explicitly enforce scope by adding braces ({}).

Pattern matching has a ways to go when compared to its functional language equivalent, but it is still a nice addition and will become more complete as the language evolves.

C# 7 Additions – Literals

A small, but nice chance in C# 7 is increased flexibility in literals. Previously, large numeric constants had no separator, and it was difficult to easily read a large number. For example, if you needed a constant for the number of stars in the observable universe (1,000,000,000,000,000,000,000), you’d have to do the following:

If you hadn’t caught the error, the constant is too short, and it’s difficult to tell looking at the numbers without a separator. In C# 7, it’s now possible to use the underscore (_) in between the numbers. So the previous example now becomes much easier to read, and it is easily recognizable the number is off.

The new version adds binary constants too. Instead of writing a constant in hex, or decimal, a constant can now be written like so:

C# 7 Additions – Throw Expressions

In previous versions, throwing exceptions had certain limitations where they could be used. Although not hampering, at times it caused additional work to validate and throw an exception, and C# 7 has removed much of the developer overhead for validation and execution.

Expressions

Previously to throw an exception in the middle of an expression there were really two options:

or

It is now possible to also throw an exception in the middle of an expression. Instead of checking for null, it is possible to throw as the second condition in the Null Coalescing Operator.

It is also possible in the Conditional Operator as well.

Expression Bodied Members

C# 6 added the ability to write a method with a single statement with a “fat arrow” (=>) and the statement. What used to be

can now be condensed to:

If you need a method stub, because you don’t know how to complete the method, and it was appropriate to use an Expression Bodied Member, you were left with two possibilities as throwing an expression wasn’t allowed by the compiler.

or

The first is error prone, because if program calls the method, there is no indication that it isn’t functioning properly. (Is null an expected return or an indicator of an error?) The second is better, but it is a little cumbersome that you must convert it to a standard function just to throw the exception. C# 7 solves this inconvenience and is now possible to throw exceptions in the Expression Bodied Member.

C# 7 Additions – ref Variables

C# 7 expands the use of the ref keyword. Along with its previous use, it can now be used in return statements, and local variables can store a reference to the object as well. At first glance, the question is “What is the real difference between returning a ref variable, and setting it through an out parameter?” Previously you could set a variable passed into a function with ref (or out) to a different value. In C# 7, you can return the reference of a property, variable etc. and store that in a local variable for later use.

The following is an examples showing its expanded use.

As expected, the PersonInformation object is passed into the GetName function which returns a reference to the string property Name. This is then passed into the MakeCapitalized function which capitalizes the name “jenny” (making it “Jenny”) in the original PersonInformation object. Compare this to the example here showing how the previous version of C# would not allow the modification of the original property in the same scenario.

Classes vs Structs

If the PersonInformation is changed to be a struct (value type) instead of a class (reference type), the following code won’t work without a slight modification, but it is still completely possible.

Structs are passed by value meaning that passing a struct into a method creates a copy of it. Returning a reference to the struct’s property would return a reference to the copied struct and would go out of scope as soon as the method completes. There would be no point, and it would cause errors pointing to properties to objects which didn’t exist.

Caveats

With these new features there are some restrictions to it. Consider this. A string can be treated as an array of characters. With the new functionality, it should be possible to pass back a reference to a character location in that string and update it, because you have the reference to the character location in the string.

Fortunately, this isn’t allowed. The compiler prevents from it being a valid option, because if this were possible, it would break the string’s immutability and cause havoc with C#’s ability to intern strings.
ref string not allowed.

The compiler is also smart enough to not allow references to variables which fall out of scope. The following is also not allowed:

After the method exits someNumber no longer exists, and when another part of the application tries to access it, it won’t be available. (You could say this might not be the case if it were a reference type like a string, but it still wouldn’t matter, because all the reference has is a location to where the object is, not the actual object itself. This causes 2 problems: One, currently there is no way to get the value from the reference. Two, the object isn’t rooted, so it could still be garbage collected at any point in time.)

The compiler is also smart enough to trace the variable use through the calling methods. This is also not allowed:

C# 7 Additions – Deconstructors

C# has a new type of method, the Deconstructor. When a type implements this method type with the name of Deconstruct, multiple variables maybe directly assigned as a return type would.

The method must be named Deconstruct and have a return type of void. The parameters to be assigned all must be out parameters, and because they are out parameters with a return type of void, C# allows the Deconstruct method to be overloaded solely based on these parameters. This is how the new System.ValueTuple allows it’s properties to be assigned to separate variables without assigning each one individually.

Deconstruct also does not need to be directly attached to the class. C# allows the method to be implemented as an extension method as well.

At the moment it is uncertain if wildcards will be added allowing unneeded variables to be omitted from being assigned. This addition would allow the insertion of the * to indicate a parameter is not needed (similar to _ in F#)

C# 7 Additions – Local Functions

In C# 7 it is now possible to create a function within a function termed a Local Function. This is for instances where a second function is helpful, but it’s not really needed in the rest of the class. It’s created just like regular functions except in the middle of another function.

Just like normal functions, you can create expression bodied members as well

Local variables in the outer functions are accessible, and it’s possible to embed local functions inside other local functions:

So how does it work? Looking at the IL code, the compiler has converted the internal function into a private static one inside the class.

IL Code showing private static function

The name is generated at compile time, so it is not accessible to other methods, but it is still possible to access it through reflection with the private and static binding flags.

reflection shows local function.

Someone I know asked what would be a good use case of Local Functions vs. Lambdas. Lambdas can’t contain enumerators, and by encasing an enumerations in a local function it allows others parts of the outer method to be eagerly evaluated. For example, if you have a method which takes a parameter and returns an enumeration, the evaluation of the parameter won’t occur until program starts to enumerate the collection. Encapsulating the enumeration in a local function allows the other parts of the outer function to be eagerly evaluated. You can find an example of the difference between using one and not using one here.

C# 7 Additions – Tuples

In C# 7 Microsoft has introduced an updated Tuple type. It has a streamlined syntax compared to it’s predecessor making it fall it look more like F#. Instead of declaring it like previous versions, the new Tuple type looks like:

Likewise to declare it as a return type, the syntax is similar to declaring it:

The first thing to note about the new type is that it is not included automatically in a new project. If you immediately use it, you’ll see the following error.

As of VS 15 preview 4 (not to be confused with VS 2015), you must include the System.ValueTuple Nuget package to take advantage of it.

This raises the question about how the new Tuple type and the previous one included since .NET 4 are related? They’re not. They are treated as two different types and are not compatible with each other.  System.Tuple is a reference type and System. ValueTuple is a value type.

So what are advantages over the previous version? The syntax simpler, and there are several other advantages.

Named Properties

In the System.Tuple version, properties of the return object were referenced as Item1, Item2 etc. This gets confusing when there are multiples of the same type in the Tuple as you have to know what position had which value type.

Now it’s possible to explicitly name the item types to reduce confusion.

The Item properties (Item1, Item2, etc.) have also been included allowing methods to be updated to the new type without breaking code based on it’s predecessor.

It’s also possible to explicitly name the values when creating the object:

Deconstruction

It is now possible to name and assign variable values upon creating (or returning) a tuple. Although not necessary, it reduces the amount of code necessary to pull values out of the type.

It’s not certain if C# will get wildcards like F# to automatically discard values which aren’t needed. If they are allowed then it’s possible to only create a variable for the name like so:

Updating Values

System.Tuple is immutable.  Once created it’s not possible to update any of the values.  This restriction has been removed in the new version.  From a purely functional perspective this could be considered a step backwards, but in C# many people find this approach more forgiving and beneficial.

Like all value types, when it is passed into a method, a copy of the tuple is created, so modifying it in the in the method does not affect the original.

However if you compare two different tuples and they have the same values, the Equals method compares the values in each and if they are all equal, it considers them equal.

Integrations with Other Languages

Unfortunately, C#’s new tuple type doesn’t automatically allow it to translate tuples from F#.

F# can’t desconstruct the values like it can with it’s native tuples, and to return it, you have to explicitly instantiate the object type and add the values.

Either way, the translation to F# isn’t horrible as it acts like any other object passed to it by C#.

It’s OK, My eval is Sandboxed (No It’s Not)

The idea of using eval has always been in interesting debate. Instead of writing logic which accounts for possibly hundreds of different scenarios, creating a string with the correct JavaScript and then executing it dynamically is a much simpler solution. This isn’t a new approach to programming and is commonly seen in languages such as SQL (stored procedures vs. dynamically generating statements). On one hand it can save a developer an immense amount of time writing and debugging code. On the other, it’s power is something which can be abused because of it’s high execution privileges in the browser.

The question is, “should ever be used?” It technically would be safe if there is a way of securing all the code it evaluates, but this limits its effectiveness and goes against its dynamic nature. So with this, is there a balance point where using it is secure, but also flexible enough to warrant the risk?

For example purposes, we’ll use the following piece of code to show the browser has been successfully exploited: alert(‘Be sure to drink your Ovaltine.’); If the browser is able to execute that code, then restricting the use of eval failed.

In the most obvious example where nothing is sanitized executing the alert is trivial:

eval will treat any input as code and execute it. So what if eval is restricted to only execute which will correctly evaluate to a complete statement?

Nope, this still successfully executes. In JavaScript all functions return something, so calling alert and assigning undefined to total is perfectly valid.

What about forcing a conversion to a number?

This still executes also, because the alert function fires when it is parsed and its return value is converted to a string and then parsed.

The following does stop the alert from firing,

But this is rather pointless, because eval isn’t necessary. It’s much easier to assign the value to the total variable directly.

What about overriding the global function alert with a local function?

This does work for the current scenario. It overrides the global alert function with the local one but doesn’t solve the problem. The alert function can still be called explicitly from the window object itself.

With this in mind, it is possible to remove any reference to window (or alert for that matter) in the code string before executing.

This works when the word ‘window’ is together, but the following code executes successfully:

Since ‘win’ and ‘dow’ are separated, the replacement does not find it. The code works by using the first eval to join the execution code together while the second executes it. Since replace is used to remove the window code, it’s also possible to do the same thing to eval like so:

That stops the code from working, but it doesn’t stop this:

It is possible to keep accounting for different scenarios whittling down the different attack vectors, but this gets extremely complicated and cumbersome. Further more, using eval opens up other scenarios besides direct execution which may not be accounted for. Take the following example:

This code bypasses the replace sanitations, and it’s goal wasn’t to execute malicious code. It’s goal is to replace the JSON.parse with eval and depending on the application might assume that malicious code is blocked, because JSON.parse doesn’t natively execute rogue code.

Take the following example:

The code does throw an exception at the end due to invalid parsing, but that isn’t a problem for the attacker, because eval already executed the rogue code. The eval statement was used to perform a lateral attack against the functions which are assumed not to execute harmful instructions.

Server Side Validation

A great extent of the time, systems validate user input on the server trying to ensure harmful information is never stored in the system. This is a smart idea, because removing before storing it tries to ensure everything accessing potentially harmful code doesn’t need to make certain it isn’t executing something it shouldn’t (you really shouldn’t and can’t make this assumption, but it is a good start in protecting against attacks). With eval, this causes a false sense of security, because code like C# does not handle strings the same way that JavaScript does. For example:

In the first example, the C# code successfully removed the word ‘window’, but in the second, it was unable to interpret this when presented with Unicode characters which JavaScript interprets as executable instructions. (In order to test the unicode characters, you need to place an @ symbol in front of the string so that it will treat it exactly as it is written. Without it, the C# compiler will convert it.) Worse yet, JavaScript can interpret strings which are a mixture of text and Unicode values making it more difficult to search and replace potentially harmful values.

Assuming the dynamic code passed into eval is completely sanitized, and there is no possibility of executing rogue code, it should be safe to use. The problem is that it’s most likely not sanitized, and at best it’s completely sanitized for now.

Just write it here, I’ll handle the rest

It’s pretty common knowledge in a .NET console application using the following command will produce the following result.

Console output

On the surface, this doesn’t look like it’s that helpful. Console applications can immediately output a message, but most applications don’t run in the console, and those that do, run in a process not visible to the user. However, System.Console has a method which makes it ideal for logging: Console.SetOut. Passing in a TextWriter object changes the system’s default output behavior and pushes the entries to a different location.

The following example outputs the contents to “C:\temp\logfile.txt” instead of the default console itself.

Text output

This allows the application to change the destination based on the need, and the destination can be anywhere. OverrideThe output class must inherit the TextWriter class, but this doesn’t mean that it has to write to a file, screen, or something which ultimately outputs to a stream. Override the appropriate methods in the TextWriter and handle the data in any manner necessary. (Here are two examples: 1. XmlLogWriter, 2. DbLogWriter)

The real advantage comes when writing larger applications where the solution consists of multiple projects requiring a way to communicate information about the system. When using a standard logging framework, every project must have knowledge of the framework to use it. This becomes cumbersome if core projects are used across multiple solutions, or if you want to change the logging framework at some point in time in the future. System.Console is available to all .NET assemblies by default, and by using it (and Console.Error.Write(Line)), the system components do not need to reference anything additional.

Along with this, the Console allows SetOut to be called at any time in the application’s lifetime, so it’s possible to change the output stream while the program is executing. This allows for the output to be changed on the fly should the need arise.

Compile Time Stored Procedures

(code for this project can be found here)

One of the largest problems with interacting with databases is the lack of compile time assurance that the application code and the SQL code interact correctly. Testing helps with finding the errors, but as a system grows in size and complexity it becomes both difficult and costly to find every mistake. There are pre-made solutions such as F# Type Providers or Entity Framework, but these present their own challenges such as not having the project in F# or avoiding the heavy handedness of Entity Framework.

Fortunately SQL server provides enough information about Stored Procedures in the Metadata tables that makes it possible to create a from scratch compile time safe mapping between stored procedures and application code. Combining this with
T4 Text Templates it’s possible to automatically generate all the code necessary to handle stored procedure inputs and returns.

The first part is to pull all the stored procedure data from the sys.procedures view.

I remove all the procedures starting with sp_ as this prefix is meant to designate that it is a system procedure for use by SQL Server. The three important pieces from the table are the Procedure Name (p.name), Object Identifier (Object_Id), and the Schema Name (s.name).

Select Sys Procedures

Armed with the object id it's now possible to pull all the data for a procedures parameters from the sys.parameters view.

With this and the procedure's name it's possible to construct code to call SQL Server stored procedures which is compile time safe. Simply generate a method which takes in the appropriate parameters to call the stored procedure. The last step is to generate a lookup table which translates the SQL datatypes to ones which are valid C# syntax.

Idenfitfing which parameters have a default value and be not passed to the procedure

Since stored procedure parameters can have default values and therefore aren't necessary to populate, it would be nice to be able to know which one's are available but aren't required. This is a little trickier. There is a field has_default_value which looks like it would tell you if the parameter has a default value and therefore is optional, but there is a problem in the SQL Server metadata table, and it is only correct for CLR parameters, and not SQL ones. In order to actually find out if parameter has a default value, there are two options.

  1. Use Sql Management Objects (SMO)
  2. Pull the stored procedure code using something like SELECT OBJECT_DEFINITION (PROCEDURE_OBJECT_ID)

Using SMO to pull the definition is actually relatively simple assuming you have access to the appropriate dlls from Sql Server Management Studio.

Code:

If you don’t have access to the SMO dlls, you can still use something akin to a regular expression (or some other parsing method) to pull the parameter information out from the Object_Definition. As with parsing anything, there are a number of gotchas to watch out for, but it is possible with a little bit of work.

User Defined Tables

Starting in SQL Server 2008 it is possible to pass tables as parameters into stored procedures. There is view called sys.table_types which lists all of the user defined tables and has the necessary columns to link it back to which stored procedures use it as a parameter, and retrieve its column definition.

Stored Procedure Output

To automatically generate C# code for SQL Server output, you’ll really need SQL Server 2012 and later for it to be effective. In that edition they added, the dm_exec_describe_first_result_set_for_object procedure

This procedure takes a stored procedure Object_Id and returns the column information concerning the first result set it finds.

Result Set

The great thing about this procedure is that not only can you generate classes which can automatically update their types and names based on a result set that changes during development, but the procedure will return nothing if the stored procedure is non functional because of a changes in the database. By running this this and having it map all the result sets, the ones which come up blank which should return data indicate the procedure will not work with the current schema and needs to be fixed. It quickly helps eliminate the difficulties of knowing which procedures will break during program execution.

Putting It All Together

With these procedure, all that is really necessary is something that opens a connection to the database, pulls the necessary data and create the appropriate C# files.

As an example and a quick starter I’ve created an example solution. The SQL metadata retrieval is in the Classes.tt file, and all the code C# generation is in the StoredProcedures.tt file. The database I used to generate the example code is found here, and uses the Adventure Works Database.