Notes from Webchat on serialization in .Net

Serialization in .Net

 What is serialization?

Serialization is the process of converting an object or a graph of objects into a linear sequence of bytes for either storage or transmission to another location. Deserialization is the process of taking in stored information and recreating objects from it.

What can it be used for?

Preserve the state of your application to persistent storage.
Distributed communication (Remoting and web services use serialization)
Deep copies of objects
Storing config information

Site settings using XmlSerializer
http://codebetter.com/blogs/david.hayden/archive/2005/03/01/56226.aspx

 XmlSerializerSectionHandler
http://pluralsight.com/wiki/default.aspx/Craig/XmlSerializerSectionHandler.html

 In .Net the support for serialization is in two namespaces

System.Runtime.Serialization – The classes in this namespace facilitate “runtime serialization”. The goal here is to preserve as much type information as possible and then recreate the object with the type information on Deserialization. Requires that the objects have the Serializable attribute. And the Serialization Formatter permission is required which is only granted to local apps by default.

 System.Xml.Serialization – The classes in this namespace facilitate XML serialization. The goal here is to provide a high performance mechanism to convert objects into XML. Only public fields and properties of the object are preserved. Used if communication between heterogeneous systems is required. Requires no special security permissions. Has a lot of restrictions as to what can be serialized and what cannot.

 
In runtime serialization

 Two aspects of serialization

            What needs to be serialized
            How is the serialized data formatted

 In .net we can do serialization using the Serializable attribute or by implementing ISerializable interface.

In case we choose Serializable attribute the formatters use reflection to get the data that needs to be serialized. Binary and SoapFormatters will serialize every field in the object including private and protected ones. It does not serialize fields marked with the NonSerialized attribute.

 
If the object implements ISerializable it gets complete control over the serialization and deserialization process.
ISerializable has only one method GetObjectData(), for Deserialization a special ctor is required. Which should be made protected and private if the class is sealed.

             BinaryFormatter
                        Writes the data to be serialized in binary format. Most efficient formatter but not useful for open systems.

             SoapFormatter
                        Writes the data to be serialized in the SOAP 1.1 format. Less efficient but a must if interoperability is a requirement. Used by remoting.

 StreamingContext

            An enumeration that can be used to decide what to serialize based on the context in which serialization is taking place. For example if the context is cross process then system wide handles like files etc. can be saved as is. The context can be specified in the constructor of the formatter. Contains information about the destination of the serialized message. It is passed to objects that handle their own serialization through ISerializable.

 Versioning

SerializationBinder
            For Assembly and Type resolution. Allows you to handle things like change in class name, change in namespace etc.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemruntimeserializationserializationbinderclasstopic.asp

 SerializationSurrogate

            Allows another class to handle the serialization of an object. Allows you to handle the case where there is an object that does not support serialization and cannot be inherited or wrapped. The problem here is that the surrogate receives a uninitialized object i.e. no ctors have run. Also can only access public fields and properties, has to resort to reflection to access private, protected members.

 An implementation of serialization surrogate that uses reflection
http://weblogs.asp.net/rosherove/archive/2005/03/03/384267.aspx

 Implementation of surrogates for binary serialization of datasets
http://msdn.microsoft.com/msdnmag/issues/04/10/CuttingEdge/

 http://www.topxml.com/xmlserializer/surrogateselectors.asp

 IDeserializationCallback
            Allows objects to do post serialization initialization.

 http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemruntimeserializationideserializationcallbackclasstopic.asp

 
XmlSerializer

                        Allows objects to be stored in XML format. Webservices internally use XmlSerializer to convert objects into SOAP format. Focuses on mapping .Net classes to arbitrary XML formats or formats defined via a schema. Optimized for performance. Takes the type to serialize in the ctor and uses the type information to give high performance while serializing and deserializing. Works only with public field and properties(provided they have both get and set properties), this is to facilitate working in restricted conditions like the intranet and internet.

 
Restrictions on types that can be serialized using XmlSerializer
http://www.topxml.com/xmlserializer/serializable_classes.asp

 Controlling Xml serialization via attributes
http://www.topxml.com/xmlserializer/serialization_attributes.asp

 Attrbutes that control Xml Serialization
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/html/cpconattributesthatcontrolserialization.asp

 Controlling Xml serialization using attributes
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/html/cpconcontrollingserializationbyxmlserializerwithattributes.asp

 When type is passed as ctor, it creates a type mapping table which it uses for serialization.

 XmlIncludeAttribute is applied on the base class to specify all derived types. Or we can use XmlElementAttribute on a field to specify the types of objects that we expect by setting the DataType property.

 For Arrays we can either use the XmlElement attribute in which case the root node for arrays is not used. Or we can use XmlArray and XmlArrayItem to control how array and array elements are serialized.

 Any class that implements IEnumerable or ICollection can be serialized and serialization can be controlled using XmlArray and XmlArrayItem attributes. When XmlSerializer detects a collection it does not serialize the public properties of the class.

 Xsd.exe can be used to generate .Net classes from an xsd schema

 Advanced Xml serialization

This can be used if we cannot attach attributes to the class, or it is not known beforehand.

 Global types – can be specified in the ctor by passing an array of additional types.

 XmlAttributeOverrides – again is specified via the ctor. XmlAttributeOverrides has two overloads of Add to apply attributes to type or to an element within a type. XmlAttributes class needs to be passed to Add() this contains strongly typed properties for the various attributes that can be applied.

 Choice elements

We can use an enumeration with XmlChoiceIdentifier to handle these situations.

 Namespaces

XmlSerializerNameSpaces class, and XmlNamespaceDecalaration or can be passed in the ctor of the XmlSerializer.
XmlSerializerNameSpaces used to map prefixes to namespaces.

Soap encoding using XmlSerializer

Using SoapReflectionImporter to generate TypeMapping which is passed to ctor of XmlSerialzier

Can be customized using SoapAttributeOverrides which takes a SoapAttributes corresponding to a type or to a member in the type.

Handling versioning problems with XmlSerializer, provides  events when it encounters unknown element, unknown node etc. Alternatively you can apply the XmlAnyElement or XmlAnyAttribute attribute to an array of type XmlElement and that will contain all unresolved elements and attributes.

Troubleshooting common problems with XmlSerialzier
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnxmlnet/html/trblshtxsd.asp

 
*XmlPreSerializer, XmlReflectionImporter, SoapReflectionImporter

 Serialization and remoting

            Remoting uses serialization to transport marshal-by-value objects.

 Serialization and datasets

            DataSets convert to XML by implementing the IXmlSerializable interface

 Common problems in serialization

            Delegates and serialization, solved by using the field:NonSerialized option. Versioning. Objects that cannot be serialized via XmlSerializer. Permission problems.

 What’s new in serialization in .Net 2.0

            http://msdn.microsoft.com/msdnmag/issues/04/10/AdvancedSerialization/default.aspx

             OptionalFieldAttribute

            OnDeserializing/OnDeserialized OnSerializing/OnSerialized ,
these events are only defined for BinaryFormatter. No new stuff has been added for the SoapFormatter.

 XmlFormatter class is now supporting all these Deserialization events.  It also supports two modes SharedContract and SharedType.

More info at

   http://weblogs.asp.net/cschittko/archive/2003/10/29/34414.aspx

http://www.douglasp.com/CommentView.aspx?guid=bff35b26-4739-4971-9a05-507e5aaddd7a

 SoapFormatter has been deprecated in .Net 2.0 no new features have been added.  XmlFormatter is the successor to that

 IXmlSerializable is now documented and can be used in applications to get control over xml serialization.

GetSchema(), ReadXml() and WriteXml() are the methods. XmlSchemaProvider attribute

Sample code illustrating use of IXmlSerializable

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnvs05/html/wsnetfx2.asp

            OptionalField allow us to specify a field as optional and a version in which it was added. Thus when a new version is deserialized the formatter does not thrown an exception if the type does not contain that field. The serialization events give more control.

 Example of OptionalField

http://fredrik.nsquared2.com/viewpost.aspx?PostID=174

 Links:

 How to serialize a hashtable

http://codebetter.com/blogs/geoff.appleby/archive/2004/11/13/32005.aspx

 Handling non serializable events in .Net 2.0 using VB.Net

http://www.lhotka.net/WeBlog/PermaLink.aspx?guid=8f8ae33f-f3b3-4864-a146-4f852d78783e

 And here are the code snippets I used during the chat

/* Creating deep copies using serialization
 *
[Serializable]
public class Customer
{
    public string Name;
    public Address Address = new Address();
   
    public Customer(string name)
    {
        this.Name = name;
    }

    public Customer DeepCopy()
    {
        MemoryStream stream = new MemoryStream();
        BinaryFormatter formatter = new BinaryFormatter();
        formatter.Serialize(stream, this);
        stream.Seek(0, SeekOrigin.Begin);
        Customer clone = (Customer)formatter.Deserialize(stream);
        return clone;
    }

    public Customer ShallowCopy()
    {
        return (Customer)this.MemberwiseClone();
    }
}

[Serializable]
public class Address
{
    public string Street;
    public string Pin;
    public string City;
}


[STAThread]
static void Main(string[] args)
{
    Customer obj1 = new Customer("Henry");
    obj1.Address.City = "Bangalore";
    obj1.Address.Street = "Banergatta";
    obj1.Address.Pin = "560076";

    Customer obj2 = (Customer)obj1.ShallowCopy();
    Customer obj3 = (Customer)obj1.DeepCopy();

    Debug.Assert(Object.ReferenceEquals(obj1.Name, obj2.Name));
    Debug.Assert(Object.ReferenceEquals(obj1.Address, obj2.Address));

    Debug.Assert(! Object.ReferenceEquals(obj1.Name, obj3.Name));
    Debug.Assert(! Object.ReferenceEquals(obj1.Address, obj3.Address));
   
    System.Console.ReadLine();
}

*/


/* Storing user settings
 *
public class UserSettings
{
    private Point _mainFormPosition = Point.Empty;
    public Point MainFormPosition
    {
        get{ return _mainFormPosition; }
        set{ _mainFormPosition = value; }
    }

    private Size _mainFormSize = Size.Empty;
    public Size MainFormSize
    {
        get{ return _mainFormSize; }
        set{ _mainFormSize = value; }
    }

    private float _conversionFactor = float.MinValue;
    public float ConversionFactor
    {
        get{ return _conversionFactor; }
        set{ _conversionFactor = value; }
    }
}


[STAThread]
static void Main(string[] args)
{
    UserSettings settings = new UserSettings();
    settings.ConversionFactor = 10;
    settings.MainFormPosition = new Point(100,100);
    settings.MainFormSize = new Size(640, 480);

    XmlSerializer serializer = new XmlSerializer(typeof(UserSettings));
    XmlTextWriter writer = new XmlTextWriter("settings.xml", System.Text.Encoding.UTF8);
    writer.Formatting = Formatting.Indented;
    serializer.Serialize(writer, settings);
    writer.Close();

    System.Console.ReadLine();
}

/* Output
<?xml version="1.0" encoding="utf-8"?>
<UserSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <MainFormPosition>
    <X>100</X>
    <Y>100</Y>
  </MainFormPosition>
  <MainFormSize>
    <Width>640</Width>
    <Height>480</Height>
  </MainFormSize>
  <ConversionFactor>10</ConversionFactor>
</UserSettings>
*/



/* Simple runtime serialization with SoapFormatter
[Serializable]
public class Customer
{
    public string Name;
    public ArrayList Orders = new ArrayList();

    [NonSerialized]
    public string DataThatShouldNotBeSerialized;

    public Customer(string name)
    {
        this.Name = name;
    }
}

[Serializable]
public class Order
{
    public string Product;
    public int Quantity;
    public Customer Customer;

    public Order(string product, int quantity, Customer customer)
    {
        this.Product = product;
        this.Quantity = quantity;
        this.Customer = customer;
    }
}


Customer customer = new Customer("James");
customer.DataThatShouldNotBeSerialized = "Random data";
Order order1 = new Order("Motherboard", 2, customer);
Order order2 = new Order("CPU", 2, customer);
customer.Orders.Add(order1);
customer.Orders.Add(order2);

SoapFormatter formatter = new SoapFormatter();
FileStream stream = new FileStream("customer.xml", FileMode.Create);
formatter.Serialize(stream, customer);
stream.Close();
*/


/* Custom runtime serialization with SoapFormatter
[Serializable]
public class Customer : ISerializable
{
    public string Name;
    public ArrayList Orders = new ArrayList();

    public string DataThatShouldNotBeSerialized;

    public Customer(string name)
    {
        this.Name = name;
    }

#region ISerializable Members

    public void GetObjectData(SerializationInfo info, StreamingContext context)
    {
        info.AddValue("Name", Name);
        info.AddValue("Orders", Orders);
    }

    private Customer(SerializationInfo info, StreamingContext context)
    {
        Name = info.GetString("Name");
        Orders = (ArrayList)info.GetValue("Orders", typeof(ArrayList));
    }
#endregion
}

[Serializable]
public class Order
{
    public string Product;
    public int Quantity;
    public Customer Customer;

    public Order(string product, int quantity, Customer customer)
    {
        this.Product = product;
        this.Quantity = quantity;
        this.Customer = customer;
    }
}


Customer customer = new Customer("James");
customer.DataThatShouldNotBeSerialized = "Random data";
Order order1 = new Order("Motherboard", 2, customer);
Order order2 = new Order("CPU", 2, customer);
customer.Orders.Add(order1);
customer.Orders.Add(order2);

SoapFormatter formatter = new SoapFormatter();
FileStream stream = new FileStream("customer.xml", FileMode.Create);
formatter.Serialize(stream, customer);
stream.Close();
*/


/* New serialization events in .Net 2.0
    
[Serializable]
public class Employee
{
    private string _name;
   
    public string Name
    {
        get{ return _name; }
        set{ _name = value; }
    }
   
    public Employee(string name)
    {
        _name = name;
    }
   
    [OnSerializing]
    private void OnSerializing(StreamingContext context)
    {
        Console.WriteLine("Serializing");
    }
   
    [OnSerialized]
    private void OnSerialized(StreamingContext context)
    {
        Console.WriteLine("Serialized");
    }
   
    [OnDeserializing]
    private void OnDeserializing(StreamingContext context)
    {
        Console.WriteLine("DeSerializing");
    }
   
    [OnDeserialized]
    private void OnDeserialized(StreamingContext context)
    {
        Console.WriteLine("DeSerialized");
    }
}


Employee emp = new Employee("Mark");

BinaryFormatter formatter = new BinaryFormatter();
FileStream stream = new FileStream("employee.dat", FileMode.Create);

formatter.Serialize(stream, emp);
stream.Seek(0, SeekOrigin.Begin);

emp = (Employee)formatter.Deserialize(stream);

stream.Close();
   
/*
This will output

Serializing
Serialized
Deserializing
Deserialized
*/

 
/* Simple Xml serialization
public class Employee
{
    public int ID;
    public string Name;
    public string Department;
}

Employee emp = new Employee();
emp.ID = 1001;
emp.Name = "Mark";
emp.Department = "Accounts";

XmlSerializer serializer = new XmlSerializer(typeof(Employee));
XmlTextWriter writer = new XmlTextWriter("employee.xml", System.Text.Encoding.UTF8);
writer.Formatting = Formatting.Indented;
serializer.Serialize(writer, emp);

writer.Close();

Output is
<?xml version="1.0" encoding="utf-8"?>
<Employee xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ID>1001</ID>
  <Name>Mark</Name>
  <Department>Accounts</Department>
</Employee>



[XmlRoot("employee")]
public class Employee
{
    [XmlAttribute("id")]
    public int ID;

    [XmlElement("name")]
    public string Name;

    [XmlElement("department")]
    public string Department;
}

<?xml version="1.0" encoding="utf-8"?>
<employee xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="1001">
  <name>Mark</name>
  <department>Accounts</department>
</employee>
*/