[LINQ via C# series]

C# is a functional and object-oriented programming language built by Microsoft. C# works with a family of programming frameworks crossing many platforms and devices. C# has been used by millions of people to build applications, services, mobile apps, and games, etc.

This is a latest and cross-platform C# programming book focusing on functional programming and LINQ data access, covering the concepts, practical usage, internal design and underlying theories.

This chapter introduces the basic concepts for readers who are new to .NET and C# language, and who want to catch up with the latest status. It then demonstrates how to setup coding tools on Windows, macOS and Linux, and finally gives an overview of functional programming and LINQ programming with C#.

Introducing cross-platform .NET, C# and LINQ

In 2002, C# 1.0 was initially released along with .NET Framework 1.0 on Windows platform. Since then, many functional features including LINQ has been built into C# language and .NET Framework. There are also multiple frameworks joined the .NET family, which enable C# and LINQ to work cross different platforms.

.NET Framework, C#, and LINQ

Microsoft .NET (pronounced “dot net”) Framework is a free development framework on Windows, widely used to build applications and services with simple programming model and good productivity. .NET Framework is based on Common Intermediate Language (CIL), an object-oriented assembly language. It mainly consists of Framework Class Library (FCL) and Common Language Runtime (CLR):

·        FCL is a set of built-in libraries of rich APIs implemented as classes, interfaces, and structures, etc. It is the fundamental used by .NET applications and services to access system functionality. FCL provides primitive types, exceptions, collections, I/O, threading, reflection, text processing, database access, and LINQ queries, etc.

·        CLR is the runtime environment that works like a virtual machine. All .NET applications and services are executed by CLR. CLR provides features including automatic memory management, thread management, structured exception handling, type safety, security, just-in-time (JIT) compiler, etc.

C# language (pronounced “c sharp”) is a general purpose high level language designed for .NET Framework. It is type-safe, generic, component-based, object-oriented and functional. It was standardized as ECMA 334 in 2002 and approved as ISO/IEC 23270 in 2003. Microsoft’s C# compiler is an implementation of these standards. It compiles C# code to CIL code, so that the CIL code is JIT-compiled to machine code by CLR, and the machine code is executed by CLR.

Besides C#, there are multiple high level languages supported by .NET Framework, like VB.NET, F#, etc., all of which are compiled or interpreted to CIL. C# is the most popular .NET language used by millions of people, according to Microsoft. Microsoft provides an ecosystem of tools for .NET software development, the flagship integrated development environment (IDE) is Visual Studio.

C# is designed to be an object-oriented programming language from the beginning, so that C# works seamlessly with object-oriented CIL of .NET Frameowrk, and many developers could easily get familiar with C#. C# started to have functional features in the initial release. Since then a lot of functional features has been built into the language. Function is first class citizen in C#, just like object. Now with the latest 7.3 release, it is easy and smooth to use C# for elegant and robust functional programming.

The real-world programs work with data in many different forms, like data objects in local memory, data in XML format, data stored in cloud database, etc. Traditionally, a specific programming model is required to work with each data format. For example, traditionally, querying a sequence of data objects in local memory can be very different from querying data rows from database table. For .NET languages and .NET Framework, Microsoft provides a unified functional programming model to query different data sources, that is LINQ (pronounced “link”), standing for “Language-INtegrated Query”. LINQ consists of language syntax and library APIs:

·        New keywords and new syntax are introduced to the .NET languages. C# has got important functional language features, including extension methods, lambda expression, query expression, etc.

·        New APIs are implemented in .NET FCL, including interfaces and classes to represent the data sources, query methods to implement the query logic, etc.

Microsoft implements LINQ syntax in .NET languages like C#, VB.NET, F#, as well as LINQ APIs in FCL to work with .NET objects, XML data, and database. At compile time, LINQ data queries written in native language keywords and syntax are compiled to regular calls of LINQ APIs; At runtime, the LINQ API calls are executed by CLR to get the work done with the specified data sources. The model of LINQ is extensible. The language syntax can work both built-in FCL APIs and custom APIs, which enables LINQ working with many data sources.

LINQ is rooted in Microsoft's Cω research project and was initially released as a part of .NET Framework 3.5 and C# 3.0. The following table shows the position of LINQ in the historic roadmap of .NET Framework and C# language:

.NET Core, UWP, Mono, Xamarin and Unity

After the evolution of 15+ years, .NET Framework has been a rich and mature ecosystem on Windows. Besides .NET Framework, C# also works with multiple other frameworks on many platforms. In 2016, Microsoft released .NET Core, a free, open source and cross-platform version of .NET Framework. .NET Core is essentially a fork a .NET Framework. It is also based on CIL, with class libraries called CoreFX  and a runtime called CoreCLR. .NET Core supports the same C# language, as well as fore mentioned F# and VB.NET. As the name suggests, .NET Core implements the core features of .NET Framework. So, it can be viewed as a subset of .NET Framework. It is designed to be a lightweight and high-performance framework to build applications and services on Windows, macOS, and many Linux distributions, including Read Hat, Ubuntu, CentOS, Debian, Fedora, OpenSUSE, Oracle Linux, etc., so that it works on a wide range of devices, clouds, and embedded/IoT scenarios. The following table shows .NET Core is released in much more agile iterations:

Microsoft also released Universal Windows Platform (UWP), the app model for Windows 10. UWP supports C# (as well as VB.NET, C++, JavaScript) development of Microsoft Store app, that can work cross all Windows 10 device families, including PC, tablet, phone, Xbox, HoloLens, Surface Hub, IoT, etc. UWP takes advantage of .NET Core. In Debug mode, UWP app is compiled to CIL, and runs against CoreCLR. In Release mode, UWP app is compiled to native binaries for better performance, and runs against .NET Native runtime.

Besides .NET Core and UWP, Mono (Monkey in Spanish) is another open source implementation of .NET Framework based on the ECMA standards for C# and CLR. Mono was initially released in 2004. It works cross many platforms, including Windows, macOS, most Linux distributions, BSD, Solaris, Android, iOS, and game consoles.

Based on Mono, Xamarin is a framework for building native mobile app on Windows, Android and iOS with C#. Microsoft acquired Xamarin in 2016 and has made it open source, with free edition available.

C# is also the language of Unity, a cross-platform game engine built by Unity Technologies. Unity also takes advantage of Mono to support C# games development on Windows, macOS, Linux, Android, iOS, and game consoles like Xbox, PlayStation, Wii, etc. Unity used to support UnityScript, a JavaScript-like language, and Boo language. Now UnityScript and Boo are being deprecated regarding the popularity of C#.

The following table summarizes these frameworks’ languages, base API surface, runtime for managed code, supported application models and platforms:

.NET Standard

As fore mentioned, C# has been standardized since its beginning. For years, the same C# language works on multiple frameworks, but each framework used to provide its own base API surface. Since 2016, Microsoft released .NET Standard specification to provide a consistent development experience, prevent the API fragmentation of different frameworks or platforms, and enable better code sharing. .NET Standard is a unified list of APIs, which should be implemented as base API surface by any framework in the .NET family. .NET Standard is represented by NuGet package NETStandard.Library, which has a reference assembly netstandard.dll. The latest major release of .NET Standard is 2.0. It has 32k+ APIs. It is supported since:

·        .NET Framework 4.6.1/4.6.2/4.7 (support with additional files), .NET Framework 4.7.1 (built-in support)

·        .NET Core 2.0

·        Mono 5.4

·        UWP 10.0.16299

·        Xamarin.Forms 2.4, Xamarin.Mac 3.8, Xamarin.Android 8.0, Xamarin.iOS 10.14

·        Unity 2018

The standardization of both C# language and library APIs provides great experience and productivity. C# developers now can learn one language and one set of base APIs, then develop many kinds of applications working cross many platforms and devices:

C#’s LINQ syntax is part of the language standard, and the core LINQ APIs are part of the .NET Standard. Microsoft also implemented other LINQ APIs, like LINQ to Entities with .NET Standard based library. So, LINQ is standardized too, and available on all frameworks in the .NET family. This book covers cross-platform LINQ technologies provided by Microsoft in great detail, including LINQ to Objects, LINQ to XML, Parallel LINQ, and LINQ to Entities.

Introducing this book

The goal of this book is to deliver latest, cross-platform, complete and in-depth knowledge on C# functional programming

.NET Standard is an object-oriented collection of reusable types, CIL is a object-oriented assembly language, and C# is also initially an object-oriented programming language, fully supporting encapsulation, inheritance, and polymorphism, so that .NET APIs and C# language work together seamlessly. In the meanwhile, C# also supports functional programming. As a typical example, LINQ is extensively functional. In C#, functions are first class citizens just like objects are. C# has plenty of functional features, like closure, higher-order function, anonymous function, etc. The LINQ features, like query expressions, lambda expression, etc., are also functional features instead of object-oriented features.

Functional programming is different from object-oriented programming in many aspects. Functional programming is usually more self-contained, more stateless, more immutable, more lazy, more side effects management, etc. The most intuitive difference is, functional programming is more declarative instead of imperative. It focus on describing what to do, instead of specifying the execution details of how to do. As a result, functional programming can be very expressive and productive. When working with data, as a typical example, functional LINQ queries provide the general capabilities of describing what is the query logic for different data source, rather than specifying the execution details of how todata access and query each specific data source, so that LINQ can be one powerful language to work with many data sources. Functional programming can also be more scalable. For example, when working with data using LINQ. As a C# book, it can be very easy to parallelize the workload multiple processor cores.

In C# development, object-oriented programming and functional programming live in harmony. For example, when a functional LINQ query works with data in local memory, the LINQ query actually works with CLR objects which represent the data. Also, when a LINQ query is executed, LINQ APIs are called, and the LINQ APIs can be internally implemented with imperative object-oriented programming.

This tutorial

This tutorial discusses cross-platform functional programming and LINQ programming via the latest C# 7.0 language, from real world development to underlying theories. It covers both .NET Framework (for Windows) and .NET Core (for Windows, macOS and Linux). This entire tutorial is based on the latest C# 7.3 language and the latest .NET Standards 2.0 which is supported by all frameworks. in the .NET family crossing many platforms. It covers C#’sis a one-stop book on the subject, covering functional features and of C# language, aspects of functional programming aspects, and the detailed usage and internal mechanisms of mainstream LINQ technologies for different data domains, including LINQ to Objects, Parallel LINQ, LINQ to XML, and LINQ to Entities. It alsoIt is also an in-depth book that discusses internal mechanisms and demystifies the underlying quintessential theories of functional programming and LINQ, including Lambda Calculus and Category Theory.

As an in-depth tutorial, some Target readers

Some basic understanding of programming and basic concepts of C# language is necessary. to get started with this book. The target audiencesreaders are those who want to learn C# language from a functional perspective or learn functional programming for Windows development and cross-platform developmentwith C# language, and those who want to learnmaster how to use LINQ in C# to work with different data in applications and servicesdomains productively. This tutorialbook is also for advanced audiences who want to learn the quintessence of readers who are already have some experience in functional programming and LINQ, but want to learn the internal mechanisms or learn the mathematical theories to build a deep and generalsolid understanding, on functional programming and those who wantLINQ.

After reading this book, you should be able to learn internal detailsmaster the concept and aspects, write cross-platform C# code in functional paradigm, use LINQ technologies to work with different data domains. With the knowledge of LINQ in order to internal mechanisms, you should be able use LINQ very effectively, and also be able to create your own LINQ APIs. And those quintessential theories not only build custom LINQ APIs or providersyou a very good understanding on C# functional programming and LINQ, but also greatly helps you understanding any other functional language.

Book structure

The contents31 chapters are organized as the following chaptersinto 3 distinct parts: code, data, and theories:

·        Part 1 Code - covers functional programming via C#, and fundamentals of LINQ.

o     Chapter 1CODE: Functional programming and LINQ paradigm

§  What is LINQ, how LINQ uses language to work with many different data domains.

·        Programming paradigm, imperative vs. declarative programming, object-oriented vs. functional programmingand LINQ Paradigm.

o     Chapter 1-2 Getting started: Introduce the basic concepts of .NET, C#, LINQ, gives an overview of functional programming and LINQ programming for different data domains, and warms up the basic syntax of C# language.

o     Chapter 2 C#3-14 functional programming in-depth

§  C# fundamentals for beginners.

o    C#: Aspects of functional programming via C#, including named function type, named/anonymous/, function polymorphism, local function, closure, function input/output, delegate, function group, lambda expression, higher-order function, currying, partial application, first class function, function. Function composition, LINQ query expression, covariance/contravariance, immutability, tuple, purity, asyncasynchronous function, pattern matching, etc., including how C# is processed at compile time and runtime.

·        Part 2 Data - covers how to use functionalDATA: Using Functional LINQ to workWork with different data domains in the real world, and how LINQ works internally.Data.

o     Chapter 315-18 LINQ to Objects

§  : How to use functional LINQ queries to work with objects, covering all LINQ and Ix.

o   How built-in standard queries and queries in interactive extension, how the LINQ to Objects query methods are implemented internally, how to implement useful custom LINQ queries.

o     Chapter 419 LINQ to XML

§  : How to modelingmodel XML data, andhow to use functional LINQ queries to work with XML data.

o    How, how to use the other LINQ to XML APIs to manipulate XML data.

o     Chapter 520-21 Parallel LINQ

§  : How to use parallelized functional LINQ queries to work with objects.

o     Performance analysis for , internal partitioning, and parallel/sequential LINQ queries query performance.

o     Chapter 622-25 Entity Framework/ Core and LINQ to Entities

§  : How to model database with object-relational mapping, and use functional LINQ queries to work with relational data in database.

§  How, how the C# LINQ to Entities queries are implemented to work with database.

o     Howinternally translated and executed, how to change data in database, and handle concurrent conflicts.

§  Performance tips and asynchrony.

·        Part 3 Theories - demystifies the abstract mathematics theories, which are the rationale and foundationsDemystifying the essentials of LINQFunctional Programming and functional programmingLINQ.

o     Chapter 726-28 Lambda Calculus via C#

§  : Core concepts of lambda calculus, bound and free variables, reduction (α-conversion, β-reduction, η-conversion), etc.

§  How to use lambda functions to represent values, data structures and computation, including Church Boolean, Church numbers, Church pair, Church list, and their operations.

o     Combinators and combinatory logic, including SKI combinator calculus, fixed point combinator for function recursion, etc.

o     Chapter 829-31 Category Theory via C#

§  : Core concepts of category theory, including category, object, morphism, monoid, functor, natural transformation, applicative functor, monad, and their laws.

§  How these concepts are applied in functional programming and LINQ.

o    How to manage I/O, state, exception handling, shared environment, logging, and continuation, etc., in functional programming.

This tutorial delivers highly reusable knowledge:

·        It covers C# knowledge in detail, which can be generally used in any programming paradigms other than functional programming.

·        It is a cross platform tutorial, covering both .NET Framework for Windows and .NET Core for Windows, macOS, Linux

·        It delivers LINQ usage and implementation for mainstream data domains, which also enables developer to use the LINQ technologies for other data domains, or build custom LINQ APIs for specific data scenarios.

·        It also demystifies the abstract mathematics knowledge for functional programming, which applies to all functional languages, so it greatly helps understanding any other functional languages too.

Code examples

The code examples are in the latest C# 7.3 language and based on the latest .NET Standards 2.0. The Parallel LINQ chapters have examples involving .NET Framework just for visualization purpose.

To make the code examples intuitive, explicit type is always used instead of var; in practice, you can decide which to use accordingly. To save the space and focus on the problems, namespace declaration and using statements are usually omitted in the code examples. For the same purpose, function’s argument null check is omitted by all code examples, in practice you should have null check as needed.

All code examples are available on GitHub: https://github.com/Dixin/Tutorial. If there is any issue, please feel free to file it hereat: https://github.com/Dixin/Tutorial/issues/new.

To save the space and paper, all code examples in this tutorial omit argument null check.

Starting to code

All tools, libraries, services involvedused in this tutorialbook are either free, or withhaving free option available. In theory, any text editor can be used for C# programming, but a powerpowerful tools can greatly improve the productivity. The following are the free tools provided by Microsoft:

·        Visual Studio Community Edition: the free and fully featured Visual Studio for Windows, the powerful and productive is the flagship integrated development environment (IDE) for C#/.NET and other development# with .NET Core, .NET Framework, UWP, Xamarin, etc. Its Community Edition is free for induvial developers or open source projects.

·        Visual Studio Code:Visual Studio for Mac Community Edition is the Visual Studio IDE available for macOS with the free Community Edition, supporting C# with .NET Core, Xamarin, etc.

·        Visual Studio Code is a free and rich code editor for Windows, macOS and Linux, supporting codingwith full support of C# and other languages with extensionswith .NET Core, as well as limited support of C# with .NET Framework on Windows.

·        Visual Studio for Mac: the free and sophisticated IDE for macOS, supporting development of .NET Core, Xamarin, etc.

Start codingCoding with Visual Studio (Windows)

The free Community Edition of Visual Studio can be downloaded from the Microsoft official website: https://visualstudio.com. To start C# programming with .NET Core, selectStandard, the “.NET Core cross-platform development” workload; in the installer is recommended, since .NET Core is the most lightweight framework supporting .NET Standard. To start C# programming with .NET Framework on Windows, select the “.NET desktop development” workload:. Other frameworks like UWP, Xamarin, can also be selected if needed.

This installs Visual Studio along with .NET Framework SDK/.NET Core SDK. To install the latest version of .NET Framework SDK/.NET Core SDK, follow the steps from the Microsoft official website: https://dot.net. After all installation

Go ahead with the installation. When it is done, launch Visual Studio. For .NET Core, click File => New => Project to create a new console applicationapp under Visual C#, .NET Core:

In Visual Studio’s Solution Explorer, under this application, there is a Program.cs file in this project, which has the application’sapp’s entry point Main:..

using System;

namespace ConsoleApp

{

    class Program

    {

        static void Main(string[] args)

        {

            Console.WriteLine("Hello World!");

        }

    }

}

Then right click the project, in the context menu click Properties. In the project property window, go to the Build tab, click the Advanced button, and change the languageLanguage version to latest:

Now right click the project again, in the context menu click “Manage NuGet Packages” to install the NuGet packages used in this tutorialbook:

·        FSharp.Core

·        linqtotwitter

·        Microsoft.Azure.DocumentDB.Core

·        Microsoft.EntityFrameworkCore.SqlServer

·        Microsoft.Extensions.Configuration.Json

·        Mono.Cecil

·        System.Interactive

·        System.Memory

·        System.Reflection.Emit.Lightweight

·        System.Threading.Tasks.Extensions

For .NET Framework, create a console application ofapp under Visual  WindowsC#, Windows classic desktop:

Change the language version to latest as well, and install the following packages:

·        ConcurrencyVisualizer

·        EntityFramework

·        FSharp.Core

·        linqtotwitter

·        Microsoft.Azure.DocumentDB

·        Microsoft.TeamFoundationServer.ExtendedClient

·        Mono.Cecil

·        System.Collections.Immutable

·        System.Interactive

·        System.Memory

·        System.Threading.Tasks.Extensions

Then right click the created project’s References child node, in the context menu click Add Reference…, and add the following framework assemblies:

·        System.Configuration

·        System.Transactions

ThisThe Parallel LINQ chapterchapters also usesuse a free Visual Studio extensionsextension for .NET Framework, Concurrent Visualizer, provided by Microsoft. it can be installed from Tools => Extensions and Updates….

More code files can be added under the application.

Now just press F5 to build, run and debug the applicationC# code in Visual Studio.

Coding with Visual Studio for Mac (macOS)

The free Visual Studio for Mac can be downloaded and installed from Microsoft official website: https://www.visualstudio.com/vs/visual-studio-mac. Then launch Visual Studio for Mac, click New Project button on the welcome page to create a new .NET Core console applicationStart coding under .NET Core, App:

Then right click the created project, in the context menu click Options. In the opened Project Options window, click the General tab under Build, change C# Language Version to latest:

Then right click the created project’s Dependencies child node, in the context menu click Add Packages, install the fore mentioned NuGet packages for .NET Core:

Now, just press F5 to build, run and debug the C# code in Visual Studio for Mac.

Coding with Visual Studio Code (Windows, macOS and Linux)

The free Visual Studio Code can be downloaded and installed from Microsoft official website: https://code.visualstudio.com. This tutorialbook also uses 2 extensions for Visual Studio Code: C# extension for C# programming, and mssql extension for SQL execution in the LINQ to Entities chapter. These extensions are both provided by Microsoft.

The .NET Core SDK needs toshould be installed separately, by following the steps from Microsoft official website: https://dot.net. TheWhen the installation is done, it can be verified by the dotnet –version command, which outputs the version of .NET Core SDK. To start coding, create a directory for a new console application, then go to this directory, run dotnet new console. 2 files are created, Program.cs and ConsoleApp.csproj. Program.cs is the C# code file, which is the same as above Program.cs created by Visual Studio. ConsoleApp.csproj is the project file containing the metadata and build information for this console application. Open ConsoleApp.csproj, and manually add the <LangVersion>latest</LangVersion> element to use the latest C# version:

<Project Sdk="Microsoft.NET.Sdk">

 <PropertyGroup>

    <OutputType>Exe</OutputType>

    <TargetFramework>netcoreapp2.0</TargetFramework>

    <LangVersion>latest</LangVersion>

 </PropertyGroup>

</Project>

The NuGet packages used by this tutorialbook can be added with the dotnet add package {package name} command. For the packages only available as preview, the version has to be specified: dotnet add package {package name} –version {version}.

From this directory, run code . command to start Visual Studio Code. Visual Studio Code should prompt “Required assets to build and debug are missing from ‘ConsoleApp’. Add them?”. Click Yes, Visual Studio Code should create the debug configuration files ininstall a .vscode subdirectory. Now,few required items. When it is done, just press F5 to build, run and debug the applicationC# code in Visual Studio Code.

Start coding:

Then right click the created project, click Options. In the opened project options window, click the General tab under Build, change the language version to latest:

Then right click the created project’s Dependencies child node, click Add Packages, install the fore mentioned NuGet packages:

Now, just press F5 to build, run and debug the code in Visual Studio for Mac.