Categories
Code

Implementing Transparent Encryption with NHibernate Listeners (Interceptors)

Have you ever had to encrypt data in the database? In this post, I’ll explore how using nHibernate Listeners to encrypt and decrypt data coming from and going into your database. The cryptography will be transparent to your application.

Why would you want to do this? SQL Server has encryption baked into the product. That is true, but if you are moving to the cloud and want to use SQL Azure you’ll need some sort of cryptography strategy. SQL Azure does not support database encryption.

What is an nHibernate Listener? I think of a Listener as a piece of code that I can inject into specific extensibility points in the nHibernate persistence and data hydration lifecycle.

As of this writing the following extensibility points are available in nHibernate.

  • IAutoFlushEventListener
  • IDeleteEventListener
  • IDirtyCheckEventListener
  • IEvictEventListener
  • IFlushEntityEventListener
  • IFlushEventListener
  • IInitializeCollectionEventListener
  • ILoadEventListener
  • ILockEventListener
  • IMergeEventListener
  • IPersistEventListener
  • IPostCollectionRecreateEventListener
  • IPostCollectionRemoveEventListener
  • IPostCollectionUpdateEventListener
  • IPostDeleteEventListener
  • IPostInsertEventListener
  • IPostLoadEventListener
  • IPostUpdateEventListener
  • IPreCollectionRecreateEventListener
  • IPreCollectionRemoveEventListener
  • IPreCollectionUpdateEventListener
  • IPreDeleteEventListener
  • IPreInsertEventListener
  • IPreLoadEventListener
  • IPreUpdateEventListener
  • IRefreshEventListener
  • IReplicateEventListener
  • ISaveOrUpdateEventListener

The list is extensive.

To implement transparent cryptography, we need to find the right place to encrypt and decrypt the data. For encrypting the data we’ll use IPostInsertEventListener and IPostUpdateEventListener. With these events we’ll catch the new data and the updated data going into the database. For decrypting, we’ll use the IPreLoadEventListener.

For this demonstration we’ll be using DatabaseCryptography class for encrypting and decrypting. The cryptography implementation is not important for this article.

IPreLoadEventListener

public class PreLoadEventListener : IPreLoadEventListener
{
readonly DatabaseCryptography _crypto = new DatabaseCryptography();

///
/// Called when [pre load].
///

///The event. public void OnPreLoad(PreLoadEvent @event)
{
_crypto.DecryptProperty(@event.Entity, @event.Persister.PropertyNames, @event.State);
}
}

IPreInsertEventListener

public class PreInsertEventListener : IPreInsertEventListener
{
readonly DatabaseCryptography _crypto = new DatabaseCryptography();

///
/// Return true if the operation should be vetoed
///

///The event. /// true if XXXX, false otherwise.
public bool OnPreInsert(PreInsertEvent @event)
{
_crypto.EncryptProperties(@event.Entity, @event.State, @event.Persister.PropertyNames);

return false;
}
}

IPreUpdateEventListener

public class PreUpdateEventListener : IPreUpdateEventListener
{
readonly DatabaseCryptography _crypto = new DatabaseCryptography();

///
/// Return true if the operation should be vetoed
///

///The event. /// true if XXXX, false otherwise.
public bool OnPreUpdate(PreUpdateEvent @event)
{
_crypto.EncryptProperties(@event.Entity, @event.State, @event.Persister.PropertyNames);

return false;
}
}

It’s important to note that on both IPreUpdateEventListener and IPreInsertEventListener must return false, otherwise the insert/update event will be aborted.

Now that we have the Listeners implemented we need to register them with nHibernate. I am using FluentNHibernate so this will be different if you are using raw nHibernate.

SessionFactory

public class SessionFactory
{
///
/// Creates the session factory.
///

/// ISessionFactory.
public static ISessionFactory CreateSessionFactory()
{
return Fluently.Configure()

.Database(MsSqlConfiguration.MsSql2012
.ConnectionString(c => c
.FromConnectionStringWithKey("DefaultConnection")))

.Mappings(m => m.FluentMappings.AddFromAssemblyOf())
.ExposeConfiguration(s =>
{
s.SetListener(ListenerType.PreUpdate, new PreUpdateEventListener());
s.SetListener(ListenerType.PreInsert, new PreInsertEventListener());
s.SetListener(ListenerType.PreLoad, new PreLoadEventListener());
})
.BuildConfiguration()
.BuildSessionFactory();
}

When decrypting and encrypting data at the application level it makes the data useless in the database. You’ll need to bring the data back into the application to read the values of the encrypted fields. We want to limit the fields that are encrypted and we only want to encrypt string values. Encrypting anything other that string values complicates things. There is nothing saying we can’t encrypt dates, but doing so will require the date field in the database to become a string(nvarchar or varchar) field, to hold the encrypted data, once we do this we lose the ability to operate on the date field from the database.

To identify which fields we want encrypted and decrypted I’ll use marker attributes.

Encrypt Attribute

public class EncryptAttribute : Attribute
{
}

Decrypted Attribute

public class DecryptAttribute : Attribute
{
}

To see the EncryptAttribute and the DecryptedAttribute in action we’ll take a peek into the DatabaseCryptography class.

DatabaseCryptography

public class DatabaseCryptography
{
    private readonly Crypto _crypto = ObjectFactory.GetInstance();

    ///
    /// Encrypts the properties.
    ///
    ///The entity. ///The state. ///The property names. 
    public void EncryptProperties(object entity, object[] state, string[] propertyNames)
    {
        Crypt(entity, propertyNames, s = >
        _crypto.Encrypt(s),
        state)
        ;
    }

    ///
    /// Crypts the specified entity.
    ///

    ///
    ///The entity. ///The state. ///The property names. ///The crypt.
    private void Crypt(object entity, string[] propertyNames, Func<string, string> crypt, object[] state) where T : Attribute
    {
        if (entity != null)
        {
            var properties = entity.GetType().GetProperties();

            foreach (var info in properties)
            {
                var attributes = info.GetCustomAttributes(typeof (T), true);

                if (attributes.Any())
                {
                    var name = info.Name;
                    var count = 0;

                    foreach (var s in propertyNames)
                    {
                        if (string.Equals(s, name, StringComparison.InvariantCultureIgnoreCase))
                        {
                            var val = Convert.ToString(state[count]);
                            if (!string.IsNullOrEmpty(val))
                            {

                                val = crypt(val);
                                state[count] = val;
                            }

                            break;
                        }

                        count++;
                    }
                }
            }
        }
    }

    ///
    /// Decrypts the property.
    ///
    ///The entity. ///The state. ///The property names. 
    public void DecryptProperies(object entity, string[] propertyNames, object[] state)
    {
        Crypt(entity, propertyNames, s = >
        _crypto.Decrypt(s),
        state)
        ;
    }

}

That’s it. Now the encryption and decryption of data will be transparent to the application and you can go on your merry way building the next Facebook.

Categories
Code

Missing Management Delegation Icon in IIS

It’s critical this is done first. Web deploy may not install correctly if it’s installed with the Management Service icon missing. Check IIS for the Management Delegation icon, it’ll be under the Management section.

If it’s missing run the following commands.

Windows 2012

dism /online /enable-feature /featurename:IIS-WebServerRole
dism /online /enable-feature /featurename:IIS-WebServerManagementTools
dism /online /enable-feature /featurename:IIS-ManagementService
Reg Add HKLM\Software\Microsoft\WebManagement\Server /V EnableRemoteManagement /T REG_DWORD /D 1
net start wmsvc
sc config wmsvc start= auto

Run Web Deploy.

Check to see if the icon is there. If it’s not, run web deploy again. It should be there.

Categories
Code

Calling Stored Procedures with Code First

One of the weaknesses of Entity Framework 6 Code First is the lack of support for natively calling database constructs (views, stored procedures… etc). For those who have not heard of or used Code-First in Entity Framework (EF), Code-First is simply a Fluent mapping API. The idea is to create all your database mappings in code (i.e. C#) and the framework then creates and track the changes in the database schema.

In traditional Entity Framework to call a stored procedure you’d map it in your EDMX file. This is a multi-step process. Once the process is completed a method is created, which hangs off the DataContext.

I sought to making a calling stored procedure easier. At the heart of a stored procedure you have a procedure name, N number of parameters and a results set. I’ve written a small extension method that takes a procedure name, parameters and a return type. It just works. No mapping the procedure and it’s parameters.

public static List<TReturn> CallStoredProcedure<TParameters, TReturn>(this DataContext context, string storedProcedure, TParameters parameters) where TParameters : class where TReturn : class, new()
{
IDictionary<string,object> procedureParameters = new Dictionary<string, object>();
PropertyInfo[] properties = parameters.GetType().GetProperties();

var ps = new List<object>();

foreach (var property in properties)
{
object value = property.GetValue(parameters);
string name = property.Name;

procedureParameters.Add(name, value);

ps.Add(new SqlParameter(name, value));
}

var keys = procedureParameters.Select(p => string.Format("@{0}", p.Key)).ToList();
var parms = string.Join(", ", keys.ToArray());

return context.Database.SqlQuery<TReturn>(storedProcedure + " " + parms, ps.ToArray()).ToList();
}

Usage

var context = new DataContext();

List<User> users = context.CallStoredProcedure<object,User>("User_GetUserById", new{userId = 3});
Categories
Code

Conditional Sql parameters with nHibernate

The problem is a the nHibernate’s CreateSqlQuery needs a complete sql string to be created, but you can’t create a string until you’ve evaluated the parameters. The only work around is to evaluate the conditional parameters to create the sql string to create the nHibernate session and then revaluate the parameters again to add them to the nHibernate query object. The problem with this, is the same evaluation logic is written twice. What is needed is a simple fluent api that will do everything for you and spit out the ISQLQuery when it’s done.

Before

public IList<AppointmentScheduleSummary> FillQuantityOld(DateTime? fromDate, DateTime? toDate, string CompanyID)
{
    string sql = "select VA.Userid as ID, E.FirstName + ' ' + E.LastName as Name,VA.Userid,count(*) as Total, getdate() as Date  from V_AppointmentScheduleStat VA, Appointment A, Employee E, Office O where";

    if (fromDate.HasValue)
    {
        sql += "  VA.Date >= '" + fromDate.Value.ToShortDateString() + "' and";

    }

    if (toDate.HasValue)
    {
        sql += "  VA.Date <= '" + toDate.Value.AddDays(1).ToShortDateString() + "' and";
    }

    sql += "  VA.date = A.date  and VA.UserId = E.UserId and O.OfficeNum = A.OfficeNum ";
    sql += " and A.appttypeid is not null";
    sql += " and O.CompanyID='" + CompanyID + "'";
    sql += " group by E.FirstName + ' ' + E.LastName ,VA.Userid  ";

    ISQLQuery query = _NHibernateSessionManager.GetSession().CreateSQLQuery(sql)
     .AddEntity("K", typeof(AppointmentScheduleSummary));
    return query.List<AppointmentScheduleSummary>();
}

After

public IList<AppointmentScheduleSummary> FillQuantity(DateTime? fromDate, DateTime? toDate,string CompanyID)
{
   var query = _NHibernateSessionManager.GetSession()
        .SQLQuery("select VA.Userid as ID, E.FirstName + ' ' + E.LastName as Name,VA.Userid,count(*) as Total, getdate() as Date  from V_AppointmentScheduleStat VA, Appointment A, Employee E, Office O where")

        .If(fromDate.HasValue, "VA.Date >= :fromDate and", parameters =>
        {
            parameters.SetParameter("fromDate", fromDate.Value.ToShortDateString());
        })

        .If(toDate.HasValue, "VA.Date <=:toDate and ", parameters =>
        {
            parameters.SetParameter("toDate", toDate.Value.AddDays(1).ToShortDateString());
            parameters.SetParameterList("", new[] {2, 3, 4,});

        })

        .Sql(" VA.date = A.date and VA.UserId = E.UserId and O.OfficeNum = A.OfficeNum and A.appttypeid is not null and O.CompanyID = :companyId" +
             " group by E.FirstName + ' ' + E.LastName ,VA.Userid")
            .SetParameter("companyId", CompanyID)
        .ToQuery();

     query.AddEntity("K", typeof(AppointmentScheduleSummary));
     return query.List<AppointmentScheduleSummary>();
}

SqlStringBuilder

using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using NHibernate;

namespace IT2.DataAccess.Extensions
{
    public class SqlStringBuilder
    {
        private readonly ISession _session;
        private readonly Action<string> _logging;
        readonly StringBuilder _builder = new StringBuilder();
        readonly ParameterBuilder _parameterBuilder;

        /// <summary>
        /// Initializes a new instance of the <see cref="SqlStringBuilder" /> class.
        /// </summary>
        /// <param name="session">The session.</param>
        /// <param name="sql">The SQL.</param>
        /// <param name="logging"></param>
        public SqlStringBuilder(ISession session, string sql, Action<string> logging)
        {
            _session = session;
            _logging = logging;
            _builder.Append(sql);
            Parameters = new Dictionary<string, object>();
            ParameterList = new Dictionary<string, IEnumerable>();
            _parameterBuilder = new ParameterBuilder(this);
        }

        /// <summary>
        /// Gets or sets the parameters.
        /// </summary>
        /// <value>The parameters.</value>
        public IDictionary<string, object> Parameters { get; set; }

        /// <summary>
        /// Gets or sets the parameters.
        /// </summary>
        /// <value>The parameters.</value>
        public IDictionary<string, IEnumerable> ParameterList { get; set; }

        /// <summary>
        /// To the query.
        /// </summary>
        /// <returns>IQuery.</returns>
        public ISQLQuery ToSqlQuery() 
        {
            string sql = _builder.ToString();

            if (_logging != null)
            {
                _logging(sql);
            }

            var query = _session.CreateSQLQuery(sql);

            foreach (var parameter in Parameters)
            {
                query.SetParameter(parameter.Key, parameter.Value);
            }

            foreach (var parameter in ParameterList)
            {
                query.SetParameterList(parameter.Key, parameter.Value);
            }

            return query;
        }

        /// <summary>
        /// To the query.
        /// </summary>
        /// <returns>IQuery.</returns>
        public IQuery ToQuery()
        {
            string sql = _builder.ToString();

            if (_logging != null)
            {
                _logging(sql);
            }

            var query = _session.CreateQuery(sql);

            foreach (var parameter in Parameters)
            {
                query.SetParameter(parameter.Key, parameter.Value);
            }

            foreach (var parameter in ParameterList)
            {
                query.SetParameterList(parameter.Key, parameter.Value);
            }

            return query;
        }

        /// <summary>
        /// Ifs the specified evaluation.
        /// </summary>
        /// <param name="evaluation">if set to <c>true</c> [evaluation].</param>
        /// <param name="sql">The SQL.</param>
        /// <returns>ParameterBuilder.</returns>
        public ParameterBuilder If(bool evaluation, string sql)
        {
            return If(evaluation, sql, null);
        }

        /// <summary>
        /// Conditionals the specified evaluation.
        /// </summary>
        /// <param name="evaluation">if set to <c>true</c> [evaluation].</param>
        /// <param name="sql">The SQL.</param>
        /// <param name="parameters">The parameters.</param>
        /// <returns>SqlStringBuilder.</returns>
        public ParameterBuilder If(bool evaluation, string sql, Action<ParameterBuilder> parameters)
        {
            if (evaluation)
            {
                _builder.Append(string.Format(" {0} ", sql));

                if (parameters != null)
                {
                    parameters(_parameterBuilder);
                }
            }

            return _parameterBuilder;
        }

        /// <summary>
        /// Sets the parameters.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="key">The key.</param>
        /// <param name="value">The value.</param>
        /// <returns>ParameterBuilder.</returns>
        public ParameterBuilder SetParameter<T>(string key, T value)
        {
            _parameterBuilder.SetParameter(key, value);
            return _parameterBuilder;
        }

        /// <summary>
        /// Sets the parameter list.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="key">The key.</param>
        /// <param name="value">The value.</param>
        /// <returns>ParameterBuilder.</returns>
        public ParameterBuilder SetParameterList<T>(string key, T value) where T : IEnumerable
        {
            _parameterBuilder.SetParameterList(key, value);
            return _parameterBuilder;
        }

        /// <summary>
        /// SQLs the specified SQL.
        /// </summary>
        /// <param name="sql">The SQL.</param>
        /// <returns>IT2.DataAccess.SqlStringBuilder.</returns>
        public SqlStringBuilder Sql(string sql)
        {
            _builder.Append(string.Format(" {0} ", sql));
            return this;
        }
    }

    public class ParameterBuilder
    {
        private readonly SqlStringBuilder _builder;

        /// <summary>
        /// Initializes a new instance of the <see cref="ParameterBuilder" /> class.
        /// </summary>
        /// <param name="builder">The builder.</param>
        public ParameterBuilder(SqlStringBuilder builder)
        {
            _builder = builder;
        }

        /// <summary>
        /// Parameters the specified key.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="key">The key.</param>
        /// <param name="value">The value.</param>
        /// <returns>ParameterBuilder.</returns>
        public ParameterBuilder SetParameter<T>(string key, T value)
        {
            _builder.Parameters.Add(key, value);
            return this;
        }

        /// <summary>
        /// Parameters the specified key.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="key">The key.</param>
        /// <param name="value">The value.</param>
        /// <returns>ParameterBuilder.</returns>
        public ParameterBuilder SetParameterList<T>(string key, T value) where T : IEnumerable
        {
            _builder.ParameterList.Add(key, value);
            return this;
        }

        /// <summary>
        /// Ifs the specified evaluation.
        /// </summary>
        /// <param name="evaluation">if set to <c>true</c> [evaluation].</param>
        /// <param name="sql">The SQL.</param>
        /// <returns>ParameterBuilder.</returns>
        public ParameterBuilder If(bool evaluation, string sql)
        {
            return _builder.If(evaluation, sql);
        }

        /// <summary>
        /// Conditions the specified evaluation.
        /// </summary>
        /// <param name="evaluation">if set to <c>true</c> [evaluation].</param>
        /// <param name="sql">The SQL.</param>
        /// <param name="parameters">The parameters.</param>
        /// <returns>ParameterBuilder.</returns>
        public ParameterBuilder If(bool evaluation, string sql, Action<ParameterBuilder> parameters)
        {
            return _builder.If(evaluation, sql, parameters);
        }

        /// <summary>
        /// SQLs the specified SQL.
        /// </summary>
        /// <param name="sql">The SQL.</param>
        /// <returns>SqlStringBuilder.</returns>
        public SqlStringBuilder Sql(string sql)
        {
            _builder.Sql(sql);
            return _builder;
        }

        /// <summary>
        /// To the query.
        /// </summary>
        /// <returns>ISQLQuery.</returns>
        public IQuery ToQuery()
        {
            return _builder.ToQuery();
        }

        /// <summary>
        /// To the query.
        /// </summary>
        /// <returns>ISQLQuery.</returns>
        public ISQLQuery ToSqlQuery()
        {
            return _builder.ToSqlQuery();
        }
    }
}
Categories
Code

Crystal Reports 13 Maximum Report Processing Limit Reached Workaround

In the Visual Studio 2012 version of Crystal Reports 13 there is a threshold that throttles concurrent reports, this also includes subreports, to 75 reports across a machine. This means if there are 5 web applications on a given server all opened reports across all 5 web applications counts toward the 75 report limit.

The error manifests itself in different ways and may result in the following errors “Not enough memory for operation.” or “The maximum report processing jobs limit configured by your system administrator has been reached”.

The problem is the reports are not disposed and they continue accumulate until the 75 limit is hit. To fix this issue, the reports have to be disposed of at the earliest possible time. This sounds simple, but is not as straightforward as it seems. Depending how the reports are generated there are two scenarios: First is generating PDF’s or Excel spreadsheets and the second is using the Crystal Report Viewer. Each scenario has a different lifetime, which we need to take into account when crafting our solution.

Solution

There are two reports lifetimes we have to manage: generated reports: PDF, Excel Spreadsheet and the Crystal Report viewer.

PDF’s and Excel Spreadsheets are generated during the request. They can be disposed on the Page Unload event. The Crystal Report viewer is a bit different. It needs to span requests and is stored in session. This makes disposing the viewer reports a bit challenging, but not impossible.

Disposing the Viewer on the Page Unload event won’t work. The Viewer has paging functionality which requests each new page from the server. To get around this issue we implemented a report reference counter. Each time a report is created, it is stored in a concurrent dictionary. When a report is disposed the report is removed from the dictionary. Upon opening a type of a report we check that the user does not already have this report open, if they do, we simply dispose the existing report and open a new one in it’s place. Other opportunities to dispose the report is on Session End (the user signs out), on Application End and when navigating away from report page.

Our internal QA team tested a non fixed version of Crystal Reports. Crystal Reports fell over at around 100 concurrent connections. After applying the fix our QA team ran a load against the servers at 750 concurrent connections without any issues.

On a side note, we have encountered latency when disposing reports with multiple sub reports.

public static class ReportFactory
{
static readonly ConcurrentDictionary<string, ConcurrentDictionary<string, UserReport>> _sessions = new ConcurrentDictionary<string, ConcurrentDictionary<string, UserReport>>();

/// <summary>
/// Creates the report dispose on unload.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="page">The page.</param>
/// <returns>``0.</returns>
public static T CreateReportDisposeOnUnload<T>(this Page page) where T : IDisposable, new()
{
    var report = new T();
    DisposeOnUnload(page, report);
    return report;
}

/// <summary>
/// Disposes on page unload.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="page">The page.</param>
/// <param name="instance">The instance.</param>
private static void DisposeOnUnload<T>(this Page page, T instance) where T : IDisposable
{
    page.Unload += (s, o) =>
    {
        if (instance != null)
        {
            CloseAndDispose(instance);
        }
    };
}

/// <summary>
/// Unloads the report when user navigates away from report.
/// </summary>
/// <param name="page">The page.</param>
public static void UnloadReportWhenUserNavigatesAwayFromPage(this Page page)
{
    var sessionId = page.Session.SessionID;
    var pageName = Path.GetFileName(page.Request.Url.AbsolutePath);

    var contains = _sessions.ContainsKey(sessionId);

    if (contains)
    {
        var reports = _sessions[sessionId];
        var report = reports.Where(r => r.Value.PageName != pageName).ToList();

        foreach (var userReport in report)
        {
            UserReport instance;

            bool removed = reports.TryRemove(userReport.Key, out instance);

            if (removed)
            {
                CloseAndDispose(instance.Report);
            }
        }
    }
}

/// <summary>
/// Gets the report.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <returns>ReportClass.</returns>
public static T CreateReportForCrystalReportViewer<T>(this Page page) where T : IDisposable, new()
{
    var report = CreateReport<T>(page);
    return report;
}

/// <summary>
/// Creates the report.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="page">The page.</param>
/// <returns>``0.</returns>
private static T CreateReport<T>(Page page) where T : IDisposable, new()
{
    MoreThan70ReportsFoundRemoveOldestReport();

    var sessionId = page.Session.SessionID;

    bool containsKey = _sessions.ContainsKey(sessionId);
    var reportKey = typeof(T).FullName;
    var newReport = GetUserReport<T>(page);

    if (containsKey)
    {
        //Get user by session id
        var reports = _sessions[sessionId];

        //check for the report, remove it and dispose it if it exists in the collection
        RemoveReportWhenMatchingTypeFound<T>(reports);

        //add the report to the collection

        reports.TryAdd(reportKey, newReport);

        //add the reports to the user key in the concurrent dictionary
        _sessions[sessionId] = reports;
    }
    else //key does not exist in the collection
    {
        var newDictionary = new ConcurrentDictionary<string, UserReport>();
        newDictionary.TryAdd(reportKey, newReport);

        _sessions[sessionId] = newDictionary;

    }

    return (T)newReport.Report;
}

/// <summary>
/// Ifs the more than 70 reports remove the oldest report.
/// </summary>
private static void MoreThan70ReportsFoundRemoveOldestReport()
{
    var reports = _sessions.SelectMany(r => r.Value).ToList();

    if (reports.Count() > 70)
    {
        //order the reports with the oldest on top.
        var sorted = reports.OrderByDescending(r => r.Value.TimeAdded);

        //remove the oldest
        var first = sorted.FirstOrDefault();
        var key = first.Key;
        var sessionKey = first.Value.SessionId;

        if (first.Value != null)
        {
            //close and depose of the first report
            CloseAndDispose(first.Value.Report);

            var dictionary = _sessions[sessionKey];
            var containsKey = dictionary.ContainsKey(key);

            if (containsKey)
            {
                //remove the disposed report from the collection
                UserReport report;
                dictionary.TryRemove(key, out report);
            }
        }

    }
}

/// <summary>
/// Removes the report if there is a report with a match type.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="reports">The reports.</param>
private static void RemoveReportWhenMatchingTypeFound<T>(ConcurrentDictionary<string, UserReport> reports) where T : IDisposable, new()
{
    var key = typeof(T).FullName;
    var containsKey = reports.ContainsKey(key);

    if (containsKey)
    {
        UserReport instance;

        bool removed = reports.TryRemove(key, out instance);

        if (removed)
        {
            CloseAndDispose(instance.Report);
        }

    }
}

/// <summary>
/// Removes the reports for session.
/// </summary>
/// <param name="sessionId">The session identifier.</param>
public static void RemoveReportsForSession(string sessionId)
{
    var containsKey = _sessions.ContainsKey(sessionId);

    if (containsKey)
    {
        ConcurrentDictionary<string, UserReport> session;

        var removed = _sessions.TryRemove(sessionId, out session);

        if (removed)
        {
            foreach (var report in session.Where(r => r.Value.Report != null))
            {
                CloseAndDispose(report.Value.Report);
            }
        }
    }
}

/// <summary>
/// Closes the and dispose.
/// </summary>
/// <param name="report">The report.</param>
private static void CloseAndDispose(IDisposable report)
{
    report.Dispose();
}

/// <summary>
/// Gets the user report.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <returns>UserReport.</returns>
private static UserReport GetUserReport<T>(Page page) where T : IDisposable, new()
{
    string onlyPageName = Path.GetFileName(page.Request.Url.AbsolutePath);

    var report = new T();
    var userReport = new UserReport { PageName = onlyPageName, TimeAdded = DateTime.UtcNow, Report = report, SessionId = page.Session.SessionID };

    return userReport;
}

/// <summary>
/// Removes all reports.
/// </summary>
public static void RemoveAllReports()
{
    foreach (var session in _sessions)
    {
        foreach (var report in session.Value)
        {
            if (report.Value.Report != null)
            {
                CloseAndDispose(report.Value.Report);
            }
        }

        //remove all the disposed reports
        session.Value.Clear();
    }

    //empty the collection
    _sessions.Clear();
}

private class UserReport
{
    /// <summary>
    /// Gets or sets the time added.
    /// </summary>
    /// <value>The time added.</value>
    public DateTime TimeAdded { get; set; }

    /// <summary>
    /// Gets or sets the report.
    /// </summary>
    /// <value>The report.</value>
    public IDisposable Report { get; set; }

    /// <summary>
    /// Gets or sets the session identifier.
    /// </summary>
    /// <value>The session identifier.</value>
    public string SessionId { get; set; }

    /// <summary>
    /// Gets or sets the name of the page.
    /// </summary>
    /// <value>The name of the page.</value>
    public string PageName { get; set; }
}
}
Categories
Code

Considerations When Throwing Exceptions

A co-worker sent an email with some code he’s struggling with. He’s trying to avoid using try/catches to drive business logic.

The problem is not the try/catches it’s simply a symptom of the problem. Can you spot the problem? You’ll have to make some assumption, but I have faith you’ll come to the same conclusion I came too.

The code is below; I changed it to protect the innocent:

private Customer GetOrCreateCustomer(long customerTelephoneNumberOrCustomerId)
        {
           Customer customer;
            try
            {
                customer = this.DoMagic(customerMasterTelephoneNumberOrCustomerId);
            }
            catch (DataException)
            {
                try
                {
                    //TODO: I know this isn't ideal. Still thinking of a better way to do this. 
                    customer = this. GetCustomer(customerMasterTelephoneNumberOrCustomerId);
                }
                catch (DataException)
                {
                    customer = this.GetCustomerFromExternal(customerMasterTelephoneNumberOrCustomerId);
                    customer.CustomerId = this.CreateCustomer(customer);
                }
            }

            return customer;
        }

There is an underlining philosophy in this system that nulls are bad. In most cases where a null can be generated an exception is thrown. At first I did not see a problem with this. I saw it as an architecture decision, an aesthetic, but as I interface with the code, it’s apparent to me it’s an architectural mistake.

You might ask, why is throwing an exception in the case of nulls bad?

Below are some guidelines when considering throwing an exception:

  1. The fact that you have to check for the null to throw the exception should be a hint that it is not needed. It an expected outcome, thus not an exception.

  2. Throwing an exception is a resource intensive operation, one of the most resource intensive operations that can be done in .Net.

  3. An exception is just that, an exception. It’s an exception to the assumptions made in the code – when these assumptions are broken, the system must terminate, it cannot move on because the system is in an unknown state (i.e. the database is no longer available) this could also be an attack vector.

  4. Throwing an exception means you have to wrap the upstream call in a try/catch block to enforce business rules. A null value is a business opportunity to control the flow of the application. The action upon the null value should be done at the point in which a business decision must take place. For example, a customer variable is null, at the UI layer a message is shown to the user stating the customer with id ‘1234’ cannot be found.

Categories
Article Code

Setting up Single Sign On with Windows 2012 and ASP.Net MVC 4

Requirements

  • Windows 2012
  • AD FS 2.0 Feature installed
  • ASP.Net MVC 4.0
  • Valid SSL certs
  • Visual Studio 2012
    This document covers setting up an ASP.NET MVC 4.0 application using Visual Studio 2012, Windows 2012 and AD FS 2.0 to enable Web Single Sign On.

It’s important to have valid SSL certificates. Self signed certificates will not work. SSL certificates are used to encrypt the tokens and will not work with self-signed certificates. This is very important. If you do not have valid certificates this will not work. Don’t waste your time without valid certificates.

Setting up AD FS 2.0 on Windows 2012

Assuming Windows 2012 is installed. With a valid SSL certificate install the SSL certificate in IIS. This is done by opening IIS (7+) Management Console, selecting the root web server node and opening the Server Certificates found in the Feature view. Do not install the AD FS feature before installing the certificate. AD FS extracts the host name from the SSL certificate and will use localhost if a certificate is not found.

If AD FS is installed before installing the SSL certificates

If this does happen, you’ll need to uninstall the the AD FS role and manually delete the IIS applications (removing them from the IIS management console is not enough they must be removed from the IIS metabase via the command line).

C:WindowsSystem32inetsrvappcmd.exe delete app "Default Web Site/adfs"
C:WindowsSystem32inetsrvappcmd.exe delete app "Default Web Site/adfs/ls"

Once a valid certificate is installed the AD FS role can be installed.

When reinstalling ADFS, the relying party needs to rebind to AD FS’s FederationMetadata otherwise you’ll encounter an ASP.Net error stating there was a token error.

Federation Services URL

At times this url uses localhost as the host. I am not certain of the cause, but unless localhost is the domain used in the ssl cert, it will not work. The following link describes how to change it.
http://technet.microsoft.com/en-us/library/dd353709%28v=ws.10%29.aspx

At this point AD FS is setup on the server. However, it does not have any trusts established, with the exception of Active Directory (configured by default). The next step is to create a trust with a Relying party.

A trust is a relationship setup between the relying party(ASP.Net MVC) and the issuer (AD FS). The trust is setup on both the relying party and the issuer.

https://myhostname.com/federationmetadata/2007-06/federationmetadata.xml

Creating a Relying Party

Assuming you have installed Visual Studio 2012, download the Windows Identity Foundation Identity and Access Tool Extension tools. Once installed create a MVC 4 project. The option “Identity and Access” is added to the project right-click menu. The Windows Identity Foundation SDK might also be a requirement.

This brings up the Identity and Access option screen.

Enter the path to the STS metadata document

The STS metadata document is generated by the AD FS server. The FederationMetadata.xml defines the Issuer (sometimes referred to as the Identity Server) and allows MVC 4(Relying Party) to create a trust between itself and the Issuer.

Enter the Realm for your Application

The realm is the MVC application. Unless the MVC application and the AD FS are on the same server the localhost host will not work. The realm is the url to your site.

Setting up a Relying Party Trust in AD FS

The Relying Party is created, now it’s time to set up the Relying Party Trust in AD FS. Back on the server open up AD FS MMC screen and click on “Add Relying Party Trust…”

The next screen asks for the FederationMetadata.xml from the relying party. In the previous step when the FederationMetadata.xml was added into the MVC 4 application a FederationMetadata.xml was created for the MVC 4 application. Now we must import the Relying Party FederationMetadata.xml into the AD FS server to complete the trust.

The FederationMetadata.xml can either be imported via a url or added via a local file. The default FederationMetadata.xml path is https://myhostname.com/FederationMetadata/2007-06/FederationMetadata.xml (same as it was for the AD FS server). If it successfully retrieves the FederationMetadata.xml you can click next until the end.

The next step is to add an endpoint. The endpoint is where a user is redirected to upon successful authentication. Start by clicking the Relying Party Trusts folder, you’ll see the Relying Party Trust that was just created. Right-click on it and select “Properties” a tabbed interface will appear. From the monitoring tab Uncheck “Automatically update relying party” this feature does not work out of the box.

Note: If the relying party is configured correctly this step is not necessary.

Click on the endpoint tab and click add. The “Add an Endpoint” dialog will appear. Select WS-Federation for the Endpoint Type. This will automatically set the Binding dropdown to “POST”. In the URL field enter the url in which an authenticated user is sent.

That’s it! The AD FS single sign on should prompt you for credentials. Once entered it will redirect you back to your site’s landing page.

Accessing AD FS with a non-IE browser

Out of the box you’ll encounter a dialog asking for username and password. For whatever reason it would not accept my domain credentials.

In a nutshell a property ExtendedProtectionTokenCheck is set to required when running windows 7. Only IE supports this feature.

More information on the issue
http://stackoverflow.com/questions/6309210/ntlm-authentication-to-ad-fs-for-non-ie-browser-without-extended-protection-sw

How to disable it
http://social.technet.microsoft.com/wiki/contents/articles/1426.ad-fs-2-0-continuously-prompted-for-credentials-while-using-fiddler-web-debugger.aspx

Troubleshooting Tips

1. Recompile the website and trying adding the trust for both ADFS and the web site.

2. Re-install AD FS, that had some issues also, but sometimes it getting to get a fresh start.

3. Installing ADFS Multiple times and you encounter a 503 on the FederationServerServices.asmx

Try removing the ACLs

netsh http delete urlacl url=http://+:80/adfs/services/
netsh http delete urlacl url=https://+:443/adfs/services/
netsh http delete urlacl url=https://+:443/FederationMetadata/2007-06/
netsh http delete urlacl url=https://+:443/adfs/fs/federationserverservice.asmx/

Continuing with the tips

4. When reinstalling ADFS after IIS was removed, the previous ADFS web directory under C:inetpubadfs* needs to be deleted.

5. The configuration service URL 'net.tcp://localhost:1500/policy' may be incorrect or the AD FS 2.0 Windows Service is not running.

setspn -l myservername

before

The SPN is not set or is incorrect. The following thread discusses it in more detail.

http://social.technet.microsoft.com/Forums/en-US/winserverDS/thread/cd9bc625-49f3-499b-9bf3-4ef32fbf64ec/

hint: casing does matter.

After

6. The X.509 certificate CN=ADFS Signing - mydomain.com is not in the trusted people store. The X.509 certificate CN=ADFS Signing - mydomain.com chain building failed. The certificate that was used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider.

This is not recommended for going into production, but it will get you past the issue:

http://social.technet.microsoft.com/wiki/contents/articles/windows-identity-foundation-wif-how-to-change-certificate-chain-validation-settings-for-web-applications.aspx

In Conclusion

My overall experience with AD FS 2.0 and Windows 2012 has been painful. Working with AD FS 2.0 reminds me of working with Sharepoint 2010 in the early days.

Some of the configuration is done differently on the Windows 2012 server versus in Visual Studio, even though you are doing the same exact things (i.e. setting up a trust). Once the server is setup and the trusts are established and working correct it’s a thing of beauty, until you get there, good luck!

Categories
Code

Creating a Generic Type at Runtime

I have a type (at runtime), I want to use it with an IOC container (in this case StructureMap) to find a generic implementation using this type. How do I do that?

It’s simpler than you’d think:

Type genericType = typeof (AbstractValidator);
Type type = genericType.MakeGenericType(bindingContext.ModelType);

//Structure Map container
var instance = _container.GetInstance(type);
Categories
Code

Deploying with MsDeploy Outside of Visual Studio

Building the msdeploy package with MSBuild.

This requires MsDeploy to be installed on the build machine.

MSBUILD /T:Package /P:Configuration=QA;PackageLocation="C:\Build\Artifacts\eserve\DEV\QA\QA.zip"

Deploying the package with MsDeploy to a web site

How to get the msdeploy command.

-source:package='C:BuildArtifactseserveDEVQAQA.zip' -dest:auto,ComputerName='https://eserve-dev.sacda.org:8172/MsDeploy.axd?site=eserve-dev',UserName='conwayc',Password='austin_1',IncludeAcls='False',AuthType='Basic' 
-verb:sync 
-disableLink:AppPoolExtension 
-disableLink:ContentExtension 
-disableLink:CertificateExtension 
-allowUntrusted 
-retryAttempts=2

Copying the package with ROBOCOPY

Copying the package to another folder with robocopy has an issue. Robocopy uses exit codes as success/error codes. CI servers look at the exit code of a command to determine success or failure. Robocopy breaks this model. Luckliy the sql team posted a code snippet to get around this issue.

rem http://weblogs.sqlteam.com/robv/archive/2010/02/17/61106.aspx
robocopy %*
rem suppress successful robocopy exit statuses, only report genuine errors (bitmask 16 and 8 settings)
set/A errlev="%ERRORLEVEL% &amp; 24"
rem exit batch file with errorlevel so SQL job can succeed or fail appropriately
exit/B %errlev%

Deploying from folder to site

-verb:sync -source:contentPath=C:BuildArtifactsSSOClientDEV -dest:contentPath="C:inetpubadfsls",computerName='http://customer.dev.myconsolidated.net
/MsDeployAgentService',userName=ccadmin,password=$urewest123

Change App Path at Commandline via MSBuild

/T:Package 
/P:Configuration=DEV;PackageLocation="C:\BuildArtifacts\Grover\Dev\Builds\DEV\Grover.zip";DeployIISAppPath=dev.grover.winnemen.com

Using MsBuild to deploy contents to folder

/T:PipelinePreDeployCopyAllFilesToOneFolder /P:Configuration=QA;_PackageTempDir="C:Build\Artifacts\Momntz\DEV\Builds\QA

Deploying Local with MSDeploy

"C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package='C:\BuildArtifacts\AlSupport.zip' -dest:auto,computerName='localhost' -allowUntrusted -retryAttempts=2 -verbose

Deploying folder to Azure with MSDeploy

The following command line is for deploying a folder to windows azure websites.

"C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:contentPath="C:\TeamCity\buildAgent\work\d018513aed1c09f\Build" -dest:contentPath="groverqa",wmsvc=waws-prod-bay-005.publish.azurewebsites.windows.net/msdeploy.axd?site=groverqa,userName=$groverqa,password=secret,authtype='Basic' -AllowUntrusted
Categories
Article Code

Code Refactor

On a recent project, I was tasked with refactoring large parts of a web system. It’s written in C#. Over time some of the code-behind files had grown to 4000 lines. The goal was to get this number down to a more maintainable level.

Over the next few posts, I’ve taken snippets of code that I refactored and will explain my thoughts and how I arrived at the solution.

The first code snippet:

    string tmp = Request.QueryString["st"];
    _varStartRecNum = tmp;
    if ((tmp != null) &amp; (!Page.IsPostBack))
    {
        _varStartRecNum = tmp;
        postBack = true;
    }

    tmp = Request.QueryString["det"];
    if ((tmp != null) &amp; (!Page.IsPostBack))
    {
        _varDetailsRecNum = tmp;
        postBack = true;
    }

    tmp = Request.QueryString["return"];
    if ((tmp != null) &amp; (!Page.IsPostBack))
    {
        postBack = true;
    }

    tmp = Request.QueryString["searchnow"];
    if ((tmp != null) &amp; (!Page.IsPostBack))
    {
        Session["selectedTab"] = "mtf";
        Session["sessionDSProviders"] = null;
        Session["mtfs"] = null;
    }

    tmp = Request.QueryString["displaywalking"];
    if (tmp == "true")
    {
        dispMtf = false;
        postBack = true;
    }

    tmp = Request.QueryString["sb"];

    if ((tmp != null) &amp; (!Page.IsPostBack))
    {
        _varSortBy = tmp;
        postBack = true;
        switch (_varSortBy)
        {
            case "Distance":
            case "Drive time":
                ddlSortBy.SelectedIndex = 0;
                break;
            case "Name":
                ddlSortBy.SelectedIndex = 1;
                break;
            case "Gender":
                ddlSortBy.SelectedIndex = 2;
                break;
            case "Clinic":
                ddlSortBy.SelectedIndex = 3;
                break;
            case "City":
                ddlSortBy.SelectedIndex = 4;
                break;
            case "Description":
                ddlSortBy.SelectedIndex = 5;
                break;
        }
    }

The above code snippet is a collection of if statements, which are an evaluation and an execution. In my first attempt, I tried to use the same evaluation for all if statements, but then I realize one was different. Not understanding the intent of the code I am forced to preserve the logic in verbatim.

Different if evaluation:

tmp = Request.QueryString["displaywalking"];
if (tmp == "true")
{
    dispMtf = false;
    postBack = true;
}

The switch statement concerned me. The condition to enter into the switch statement is the same as the others. I decided to proceed and worry about the switch statement later.

The code uses the same variable, the ‘tmp’ variable, to retrieve different query value. The value is overwritten with each query value retrieval. For clarity I create a variable for each query value:

string st = Request.QueryString["st"];
string det = Request.QueryString["det"];
string @return = Request.QueryString["return"];
string searchNow = Request.QueryString["searchnow"];
string displayWaling = Request.QueryString["displaywalking"];
string sb = Request.QueryString["sb"];

The next step was to isolate the evaluation and expression while keeping them associated with each other. If an evaluation is true, I want to execute its corresponding expression. I created a class that represented the association.

private class Evaluate
{

    public Func Evaluation { get; set; }

    public Action Expression { get; set; }
}

Now I can create an evaluation, and if it’s true, I can execute its expression.

The next problem was how to use the above class with all the if statements. I was worried the expressions might get unwieldy in a collection. The whole purpose was to create a concise scaleable solution. The existing solution was neither.

var eval = new[]
               {
                   new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(st) &amp;&amp; !IsPostBack), Expression = () =&gt; { _varStartRecNum = st;postBack = true; }},
                   new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(det) &amp;&amp; !IsPostBack), Expression = () =&gt; { _varStartRecNum = det;postBack = true; }}, 
                   new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(@return) &amp;&amp; !IsPostBack), Expression = () =&gt; {postBack = true; }}, 
                   new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(searchNow) &amp;&amp; !IsPostBack), Expression = () =&gt; {Session["selectedTab"] = "mtf";Session["sessionDSProviders"] = null; Session["mtfs"] = null;}}, 
                   new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(displayWaling)), Expression = () =&gt; {dispMtf = false; postBack = true;}}, 
                   new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(sb) &amp;&amp; !IsPostBack), Expression = () =&gt; {_varSortBy = sb;postBack = true; SetSort(_varSortBy);}}, 
               };

It turned out better than I expected. One drawback with my solution is, if you don’t know how to use delegates, you’ll be screwed when it comes to maintaining the above code.

The last stumbling block was the switch statement. It was not going to fit gracefully into my anonymous collection, but then it didn’t need to:

private void SetSort(string sortBy)
{
    switch (sortBy)
    {
        case "Distance":
        case "Drive time":
            ddlSortBy.SelectedIndex = 0;
            break;
        case "Name":
            ddlSortBy.SelectedIndex = 1;
            break;
        case "Gender":
            ddlSortBy.SelectedIndex = 2;
            break;
        case "Clinic":
            ddlSortBy.SelectedIndex = 3;
            break;
        case "City":
            ddlSortBy.SelectedIndex = 4;
            break;
        case "Description":
            ddlSortBy.SelectedIndex = 5;
            break;
    }
}

By encapsulating it into a method, I was able reference the method in the expression. It worked every nicely.

new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(sb) &amp;&amp; !IsPostBack), Expression = () =&gt; {_varSortBy = sb;postBack = true; SetSort(_varSortBy);}

The last component is iterating over the collection:

foreach (var evaluate in eval.Where(evaluate =&gt; evaluate.Evaluation()))
{
    evaluate.Expression();
}

The complete solution:

private class Evaluate
{
    public Func Evaluation { get; set; }

    public Action Expression { get; set; }
}

private void SetSort(string sortBy)
{
    switch (sortBy)
    {
        case "Distance":
        case "Drive time":
            ddlSortBy.SelectedIndex = 0;
            break;
        case "Name":
            ddlSortBy.SelectedIndex = 1;
            break;
        case "Gender":
            ddlSortBy.SelectedIndex = 2;
            break;
        case "Clinic":
            ddlSortBy.SelectedIndex = 3;
            break;
        case "City":
            ddlSortBy.SelectedIndex = 4;
            break;
        case "Description":
            ddlSortBy.SelectedIndex = 5;
            break;
    }
}

private void EvaluateQueryParameters()
{
    string st = Request.QueryString["st"];
    string det = Request.QueryString["det"];
    string @return = Request.QueryString["return"];
    string searchNow = Request.QueryString["searchnow"];
    string displayWaling = Request.QueryString["displaywalking"];
    string sb = Request.QueryString["sb"];

    var eval = new[]
                   {
                       new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(st) &amp;&amp; !IsPostBack), Expression = () =&gt; { _varStartRecNum = st;postBack = true; }},
                       new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(det) &amp;&amp; !IsPostBack), Expression = () =&gt; { _varStartRecNum = det;postBack = true; }}, 
                       new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(@return) &amp;&amp; !IsPostBack), Expression = () =&gt; {postBack = true; }}, 
                       new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(searchNow) &amp;&amp; !IsPostBack), Expression = () =&gt; {Session["selectedTab"] = "mtf";Session["sessionDSProviders"] = null; Session["mtfs"] = null;}}, 
                       new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(displayWaling)), Expression = () =&gt; {dispMtf = false; postBack = true;}}, 
                       new Evaluate {Evaluation = () =&gt; (!string.IsNullOrEmpty(sb) &amp;&amp; !IsPostBack), Expression = () =&gt; {_varSortBy = sb;postBack = true; SetSort(_varSortBy);}}, 
                   };

    foreach (var evaluate in eval.Where(evaluate =&gt; evaluate.Evaluation()))
    {
        evaluate.Expression();
    }
}

In the end, I like this solution better than the original. One of the drawbacks is the level it’s written. I wanted to create a simpler solution that any developer could maintain. There isn’t anything difficult about the above code; I’m creating a collection and iterating over it. The confusion comes in with the evaluation and the expressions. It’s not a beginner topic.