Are you Null?

Within the last couple of days Microsoft released a proposed update for the next major release of C# version 8.  Over the past several years, there has been a large debate on the existence and use of null in software development.  Allowing null has been heralded as the billion dollar mistake by the null reference inventor, Sir Tony Hare. With this, Microsoft has decided to help the C# community by adding functionality to the C# compiler to help point out where a null reference might occur.

With the release of C# 8, anything referencing an object (string, etc.) must explicitly declare itself as possibly being null, and if that variable isn’t explicitly checked before being used, the compiler generates a warning that a possible null reference might occur. So how does this work? By using the ? at the end of a reference type, it signifies the developer acknowledges null might occur.

This looks like it would be a breaking change, and all code written in a previous version will suddenly stop compiling. This would be true except for two things.

  1. You must use a compiler flag to enforce the rule.
  2. The flag will only generate warnings not errors.

So legacy code is safe in the upgrade process if it’s too difficult to convert.

With this, they are still working out a number of scenarios that prove tricky to handle. These are things like default array initialization (new string[2]). Their comments about all of these can be found on their blog on MSDN

I’ve added their code examples below of edge cases they are still working on:

Personally, I hoped the compiler would enforce these rules a little stronger. Some languages like F# strictly enforce variable immutability unless explicitly allowed, and other functional languages do not allow it at all.

It is possible to turn on “Warnings as errors” and have the compiler stop if it encounters a possible null exception, but this assumes the rest of the code has no other warnings that won’t stop compilation. Ideally, no warning flags should ever appear in code without being fixed, but that is a very difficult standard follow when dealing with legacy code from years past where no one followed that rule before you. Either way, the C# team was in a tight situation, and they did the best they could. They needed to make strides towards making null references easier to track, but they couldn’t break all of the legacy code using previous versions of C#.

Functional Languages in the Workplace

On a semi regular basis, people question why I choose to use F# to implement projects. They question why use a lesser known language when one like C# has a larger developer pool and is more widely documented. I explain to them my rational behind it, siting personal experience, and documented cases about others success stories as well. There is significant evidence showing functional languages can reduce commonly occurring defects due to their inherent nature of immutability provide easier support for scalability, and have a stronger type system allowing for more expressive code. There are numerous testimonials on the use of functional languages and their benefit, but after hearing all of this, they are still doubtful about even considering a change. Assuming this evidence is correct, the question of “Why isn’t this a serious choice for the majority of organizations?” continues to appear.
During discussions about switching to a functional language, I repeatedly hear the several common questions and arguments for resisting change. Most of these embody fear, uncertainty, and doubt. Several can be applied to moving to any technology, and although they should be considered, they are nothing which cannot be overcome. Here are my responses to the most common arguments against change I receive.

Our code is already written in language X, and it will be hard to make a change

There will always be legacy code, and it probably deviates from the standards used today. Was it written in a previous version of the currently used language? Does it contain libraries that are no longer supported? Was it written in such a way that converting it to current standards is difficult or impossible? If the answers these questions is yes, that doesn’t mean that other projects suffer the same fate.
Legacy code can’t hold you back from technological advancements, and it most likely doesn’t now. Over the last several years many software vendors have made sweeping changes to languages and technologies leaving them looking only vaguely like what they did when first created. The introduction of Generics, the inclusion of Lambda Expressions, and asynchronous additions made huge advancements in several different languages and greatly changed common approaches to solving problems. These enormous changes didn’t stop organizations from modernizing their many of their applications to take advantage of new features even though code written with them is radically different than in previously created applications.
Radical shifts in technology happen all the time, and almost every organization shifts its strategies based on trends in the industry. Organizations which defer changes to their current approach often find greater difficulty in migrating the longer they wait due to the fact that they continue to implement solutions using their current approach. Mindlessly shifting from one approach to another is never a wise decision. That introduces chaos, but neglecting trying new approaches due to legacy concerns can only end in repeating the same mistakes.

Our developers don’t know language Y. It will be too hard and costly for them to learn and migrate.

A developer’s job is to learn every day. There are new features to understand, new architecture patterns to master, and new languages to learn. The list is endless. The belief that at any stage in one’s career the road to deeper understanding ends, is myopic and ultimately an exit ramp to another profession or a stagnant career. Developer’s should be challenged. Organizations should push their staff to understand new things, and compared to the opportunity cost of repeating the same mistakes, the amount of time and money required to train people is often negligible, especially with tools like books, video learning, computer base training, etc.
There are some people that have no desire to continue learning, and that’s ok. New development isn’t for everyone, and going back to the previous point, there are always applications in need of support that won’t or can’t be converted. Organizational migration to a technology is almost never an all or nothing approach, and some solutions should be left exactly how they are, because of the cost of converting them will outweigh the benefits. There will be people to maintain those in the long term, and these solutions cannot be the bedrock against advancing how other projects progress.

What if we fail and we are stuck with a language we can’t use?

If an organization takes the leap of faith and switches to a functional language what is the probability of some failure during the process? The initial answer is, 100%. Everyone fails every day at something. Failure is inevitable. With this in mind, you’re already failing at something, so the question is what are you going to do to try and fix it? You’re going to create other problems too, but with planning, retrospective analysis, and learning from those mistakes, those will be solved as well, but ultimately the position you end at will be further along than where you started.
A few years ago, I had a discussion with an organization about their development practices. They were extremely adept at knowing where their time was allocated: support, feature enhancements, refactoring, etc. When asked about their breakdown, they explained on average 30% of their time went to fixing production defects from previous releases. They were perplexed about why they were missing deadlines despite becoming stringent on code quality. I asked about their plan to fix it, and they responded with a few ideas, but their final answer distilled to, “write better code.” When confronted with the question, “What are you going to change?” they said, “Nothing. Changing the development process is too time consuming and costly. If we update our practices, we’ll fall further behind on our releases.” The definition of insanity is doing the same thing and expecting a different result, yet several organizations believe they can break the cycle simply by standing still. If changing how an organization develops isn’t feasible, then changing what they develop with is one of the only few viable options remaining. It is much easier to change a technology than it is to change an ingrained culture, which is exactly why using languages and tools that enforce practices which reduce errors is a much more efficient approach than convincing everyone to work in a certain way.
Most organizations resistant to change perceive technology migrations as a revolutionary approach. They firmly believe all use of a certain technology immediately stops and the new one begins, because it is much easier to think in terms of black and white (one vs. the other) when change is a rare and uncomfortable occurrence. Change to anything new should be a cautious approach and take practice. It should be evolutionary. Organizations should try several smaller variations of an approach, learning from each and refining their ideas on gradually larger projects. Embracing a adaptation and “failure leads to a stronger recovery” approach ultimately leads to a better outcome.
It is almost certain moving from to a functional language from an unrelated paradigm is going to be difficult and confusing, but the fault does not lay to the language itself. As with anything new, the concepts are unfamiliar to those starting to use it. There will be mistakes during the learning process, and some projects will probably take longer than expected, but basing the long-term benefits on the first attempt to implement anything will show biased result against it, and with time moving to an approach which aids developers to make fewer mistakes and write better and cleaner code will save both time and money.

It’s not widely used enough for us to find people to support it

My coworker recently attended two meetups concerning functional programming, each having approximately 25 attendees. After the first one, he decided to do an experiment at the second. He asked people at the meetup, “How many of you use a functional language at work?” and the result was astounding. Only one person admitted to it, and it was only part time. At a minimum, there are 25 people at each location that are excited enough about functional programming to attend a meetup on their own time on a topic which has nothing to do with the tools they use at work, and these people are only a representation of the larger workforce. There are many others that were either unable to attend, or were unaware of the event.
There is almost no place in the United States that isn’t a competitive market for development staff. Large companies are able to pay higher rates and have better benefits which means they will pull the majority of the highest qualified candidates. Smaller organizations can’t offer the enormous benefits packages placing them in a difficult situation to fill needed positions. Picking a technology where there are fewer people to fill the role would seem to place those organizations at a disadvantage, but this is actuality in comparison to overall demand for those type of people. Looking solely at the number of potential applicants, the pool of functional programmers is smaller, but organizations using functional languages aren’t nearly as widespread, so they suffer less completion when searching for candidates. Furthermore, assuming the statistics surrounding the benefits of functional languages are correct, organizations will require fewer programmers accommodating the constraint of a smaller pool of applicants.

Conclusions

Functional languages can be an excellent fit for organizations, both ones starting development and others which have been established for a considerable length of time. Most resilience in using them comes from misunderstanding the benefits compared to the cost of changing languages. It is neither difficult nor time consuming to attempt to better the development process by focusing on tools to better aid the process.

Regular Expressions Presentation

I have uploaded my notes from my presentation on Regular Expressions.  Currently, I am flushing out my presentation notes into a more readable format in the readme file, but I have uploaded everything now in case you want to get the raw notes early.  They can be found at: https://github.com/kemiller2002/RegexBlackMagic/

 

Practice

When was the last time you sat down and talked to your team about problems?  What was the last task or procedure you changed because it was a bad fit for the project?  The longer you wait, the worse it gets because the longer a team works together, the less likely someone is going to mention a difficulty or a frustration.  Once people get used a routine that is bearable, they will learn to live with it even though it’s uncomfortable, and it’s this situation which leads to frustration, turnover and burnout.

I am fortunate to have a team who is vocal and willing to discuss issues.  The team is relatively good working through development scenarios where issues might arise.  Code deployments are automated, database updates are tested multiple times in varying scenarios, and every time an issue occurs that a change in process could help, we look at implementing it.  For a long time, we were diligent at applying this test and fix approach to everything except ourselves, and that is when the unexpected happened.  My team member completely shocked me and let me know that our communication with him was poor and he didn’t feel included.  He works remotely most of the time, and for the most part he was the only one.  The rest of the team would have conversations in hallways, etc., and to us this was the course of a normal day.  His knowledge and insights were being excluded, simply because we were making decisions about things we didn’t feel warranted an official meeting.

This left us in an awkward position.  We either needed to end the possibility of working remotely, or we needed to rethink how we communicated on a daily basis.  Our problem was that most of us didn’t even know we had a problem, and that certainly meant we didn’t know how to fix it.  Most organizations handle problems like this by minimizing scenarios where they have issues.  We took the opposite approach and forced ourselves to confront it, understand it, and from this we created a new policy; all people must now work remote at least one day a week.  Why did we take this approach?  We understood that remote work is too important to give up. We also understood that unless we analyzed what was wrong it would never improve.  In the end, we would rather take time to fix a problem than keep suffering through it.  This approach to helped not only communication with but showed us other areas for improvement as well.

Communication

For us to solve the problem, we must first all understand what it is.  With all people working away from the office, we can all see what problems there are with communication.  Each person can now look at how the team functions and provide a unique perspective on how to make it better.   We found that not only our communication with people off site improved, but with people on site as well.  We are now much more diligent about communicating ideas and decisions with everyone and we are much more cognizant about recording information where it can be accessed anywhere at any time.

Focus

Despite any attempt at quiet working conditions, most offices are a chaotic place.  Programming requires concentration and several tasks are only easy to accomplish when someone can have several hours of uninterrupted work.  Common pieces of advice include, “put on headphones” or “book a meeting room and close the door.”  These are fine, but there is always the possibility people will cause interruption.  While at home, coworkers cannot do this allowing greater relaxation which leads to an easier ability to focus.  Now that each team member has at least one day where work can be uninterrupted, they commonly save long tasks for when not at the office.

Disaster Recovery

Being able to access key internal systems from home is not just for people who live too far away to be on location.  We depend on it in cases when people can’t drive to the office, or when emergencies arise when we don’t have time to make the commute.  During a crisis is not the time to find out your equipment doesn’t function.  With each member testing remote access on a weekly basis, we have a relativity high certainty that it will work when we need it to.   This is a tool used in emergencies just like redundant servers, or a secondary site.  You can always hope it works when necessary, but you won’t know until you try.

Trust

Trust is something which everyone wants to believe exists but is often in short supply.  Most places have the capability to allow people to work remotely, but leaders often joke about their employees watching television instead. (I actually interviewed at a company where the hiring manager threateningly said that he’ll know if people aren’t doing work while they are remote.)  Allowing employees to work at home when necessary shows a level of implicit trust.  It tells employees management has enough faith in their work ethic that if its only once in a while the project won’t suffer too much.  Forcing someone to work remotely changes the narrative.  It becomes a common occurrence, and shows everyone they are trusted enough to do what they need to do.  Trust among a team is key.  It allows people to be open about issues and ideas for improvement, and without it teams will fail to improve.

Quirks with Pattern Matching in C# 7

With C# 7, Microsoft added the concept of pattern matching by enhancing the switch statement. Compared to functional languages (both pure and impure), this seems to be somewhat lacking in a feature by feature comparison, however it is still nice in allowing a cleaner format of code. With this, there are some interesting quirks, that you should be aware of before using. Nothing they’ve added breaks existing rules of the language, and with a thorough understanding how the language behaves their choices make sense, but there are some gotchas that on the surface looks like they should function one way, but act in a completely different manner.

Consider the following example.

Shows

C# 7 now allows the use of a switch statement to determine the type of a variable. It as also expanded the use of is to include constants including null.

is can show if something is null : shows true

With these two understandings, which line executes in the following code?

Shows default code executed.

Based on the previous examples, its a reasonable conclusion that the one of the first two case statements would execute, but they don’t.

The is operator

The is operator was introduced in C# 1.0, and its use has been expanded, but none of the existing functionality has changed. Up until C# 7, is has been used to determine if an object is of a certain type like so.

This outputs exactly as expected. The console prints “True” (Replacing string with var works the exactly the same. Remember that the object is still typed. var only tells the compiler to figure out what type the variable should be instead of explicitly telling it.)

Is Operator String: True

What happens if the string is null? The compiler thinks its a string. It will prevent you from being able to pass it to methods requiring another reference type even though the value is explicitly null.

Type is null

The is operator is a run time check not a compile time one, and since it is null, the runtime doesn’t know what type it is. In this example, the compiler could give flags to the runtime saying what type it actually is even though it’s null, but this would be difficult if not impossible for all scenarios, so for consistency, it still returns false. Consistency is key.

Printing out True and False is nice, but it’s not really descriptive. What about adding text to describe what is being evaluated.

Is Type With Question, Question doesn't appear

Why didn’t the question appear? It has to do with operator precedence. The + has a higher operator precedence than is and is evaluated first. What is actually happening is:

This becomes clear if the clause is flipped, because the compiler doesn’t know how to evaluate string when using the + operator.

Flipping clauses throws error.

Adding parenthesis around the jennysNumber is string fixes the issue, because parenthesis have a higher operator precedence than the + operator.

output of is operator and + flipped with parenthesis (shows both question and value)

Pattern Matching with Switch Statements

Null and Dealing with Types

Null is an interesting case, because as shown during the runtime, it’s difficult to determine what type an object is.

Base Example

This code works exactly as how you think it should. Even though the type is string, the runtime can’t define it as such, and so it skips the first case, and reaches the second.

Adding a type object clause works exactly the same way

shows object case works same way

What about var. Case statements now support var as a proposed type in the statement.

If you mouse over either var or the variable name, the compiler will tell you what type it is.
show compiler knows what type it is.

Shows var case statement doesn't know type

It knows what the type is, but don’t let this fool you into thinking it works like the other typed statements though. The var statement doesn’t care that the runtime can’t determine the type. A case statement with the var type will always execute provided there is no condition forbidding null values when (o != null). Like before, it still can’t determine the type inside the case statement statement.

Why determine object type at compile time?

At any point in time (baring the use of dynamic), the compiler knows the immediate type of the variable. It could use this to directly point the correct case concerning the type. If that were true, it couldn’t handle the following scenario, or any concerning inheritance of child types.

shows is string

Personally, I would like to see either a warning or an error, that it’s not possible for type cases to determine if the variable is null case string s when (s is null), but as long as the code is tested and developers knows about this edge case, problems can be minimized.

All the examples can be found on github: https://github.com/kemiller2002/StructuredSight/tree/master/PatternMatchingQuirks_Standard

Indy.Code()

I have been fortunate enough to be allowed to speak at Indy.Code() which really means I’m grateful to get the chance to listen to all the other speakers. If you live anywhere around the Midwest, or can travel to Indianapolis, I highly recommend you come and watch the talks. It’s going to be an amazing event with some unbelievably great speakers. A fun time will be had by all. (Unless you hate fun, then you probably won’t like it so much. For everyone else, it’ll be pretty great!)

https://indycode.amegala.com/