Installing Windows Identity Foundation 3.5 in Azure Role Startup Task (Server 2012 VM)

The new VMs used by Azure are Server 2012+, and you cannot use the msu installer for WIF (Like you could here: http://blogs.msdn.com/b/sriharsha/archive/2012/04/07/windows-azure-unable-to-find-assembly-microsoft-identitymodel.aspx).

You must use dism to enable the feature.

I assume you are familiar with creating startup tasks – you need to create a batch file that runs:

Dism /online /Enable-Feature /FeatureName:Windows-Identity-Foundation >> "%TEMP%\WifStartupLog.txt" 2>&1

Here is the error you might be searching for:

Could not load file or assembly ‘Microsoft.IdentityModel Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ or one of its dependencie. The system cannot fine the find specified.

Get the SQL Server and database name from Dynamics CRM

As far as I can tell, there is no way to retrieve the SQL Server name using the Organization Service. If you have the Organization Service URL, there really is no good way to get the database connection string.

(You can use the deployment service to get it, but that only works if you happen to be a deployment administrator)

No good way… but there is a way. When you download a dynamic spreadsheet, the database connection string is embedded in the xml. And you can programmatically download that spreadsheet using an entity that all CRM orgs have.

Using some fiddler experimentation, I was able to make a .NET method that will extract the database connection string, given the organization base url (https://crmserver/orgnam)

        /// <summary>
        /// Given a Dynamics CRM Org URL, retrieve the Database Connection string
        /// </summary>
        /// <param name="crmOrgUrlBase"></param>
        /// <returns></returns>
        private static string GetCrmDatabaseConnectionString(string crmOrgUrlBase)
        {
            string cleanurl = crmOrgUrlBase.ToLowerInvariant().Trim().Replace("/xrmservices/2011/organization.svc", "");
            {
                int mainPos = cleanurl.IndexOf("/main.aspx");
                if (mainPos > 0)
                {
                    cleanurl = cleanurl.Substring(0, mainPos);
                }
            }

            string requestPayload = @"xdpi=96&exportType=list&useSqlQuery=1&fetchXml=%3Cfetch+distinct%3D%22false%22+no-lock%3D%22false%22+mapping%3D%22logical%22+page%3D%221%22+count%3D%2250%22+returntotalrecordcount%3D%22true%22%3E%3Centity+name%3D%22systemuser%22%3E%3Cattribute+name%3D%22systemuserid%22%2F%3E%3Cattribute+name%3D%22fullname%22%2F%3E%3Cattribute+name%3D%22fullname%22%2F%3E%3Corder+attribute%3D%22fullname%22+descending%3D%22false%22%2F%3E%3C%2Fentity%3E%3C%2Ffetch%3E%0D%0A&layoutXml=%3Cgrid+name%3D%22excelGrid%22+select%3D%220%22+icon%3D%220%22+preview%3D%220%22%3E%3Crow+name%3D%22result%22+id%3D%22systemuserid%22%3E%3Ccell+name%3D%22fullname%22+width%3D%22100%22%2F%3E%3C%2Frow%3E%3C%2Fgrid%3E%0D%0A";
            string url = cleanurl + "/_grid/print/export_live.aspx";

            string response = null;


            using (var wc = new System.Net.WebClient())
            {
                wc.UseDefaultCredentials = true;
                wc.Headers.Add("Accept-Encoding", "gzip, deflate");
                wc.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
                wc.Headers.Add("DNT", "1");
                response = wc.UploadString(url, requestPayload);
            }


            var xe = System.Xml.Linq.XElement.Parse(response);


            System.Xml.Linq.XNamespace nn = "urn:schemas-microsoft-com:office:spreadsheet";
            System.Xml.Linq.XNamespace nn2 = "urn:schemas-microsoft-com:office:excel";


            var connectionEl = xe.Element(nn + "Worksheet").Element(nn2 + "QueryTable").Element(nn2 + "QuerySource").Element(nn2 + "Connection");

            string rawString = connectionEl.Value;

            Regex rex = new Regex("SERVER=([^;]+);DATABASE=([^;]+)", RegexOptions.Compiled);

            var m = rex.Match(rawString);

            string server = m.Groups[1].Value;
            string database = m.Groups[2].Value;

            return "DATA SOURCE=" + server +";INITIAL CATALOG=" + database + ";Integrated Security=SSPI";
        }

It is technically unsupported to access export_live.aspx directly, so this may stop working in future releases of CRM.
I have tested this on CRM 2011 UR 11 and UR 17.

Dirt simple method of limiting System.Threading.Task concurrency (max concurrent threads)

Microsoft has an article about creating a Task Scheduler that limits maximum concurrency:
How to: Create a Task Scheduler That Limits the Degree of Concurrency

The provided example class is clunky and kinda hard to grasp. I’ve been searching for a simple way to create a “thread pool” with limited concurrency. I previously experimented with ConcurrentBags of BackgroundWorkers – this worked, but it was also fairy complex and cumbersome.

The solution I came up with uses a Semaphore (or rather the lighter weight SemaphoreSlim type) to manage concurrency.

Simply put – you create a semaphore with a maxCount (and initialCount) equal to the max concurrency you desire. Then when you want to fire off a Task, first call semaphore.Wait(), then call semaphore.Relase() in a ContinueWith().

Contrived example:

    public class LimitedAsync
    {
        private SemaphoreSlim _semaphore;

        public LimitedAsync(int maxConcurrency)
        {
            // Create semaphore with maxConcurrency slots
            _semaphore = new SemaphoreSlim(maxConcurrency, maxConcurrency);
        }

        public void DoSomethingAsync(string param)
        {
            //Wait for semaphore to have availablilty (blocks if semaphore is full)
            _semaphore.Wait();
            //Run DoSomething in a task, then release slot in semaphore
            //ContinueWith is called even if DoSomething faults
            Task.Factory.StartNew(() => DoSomething(param)).ContinueWith((x) => _semaphore.Release());
        }

        public void DoSomething(string param)
        {
            System.Threading.Thread.Sleep(500);
        }
    }

Generating a single flattened WSDL from an existing WCF Service

UPDATE 4/12/2013: Tool included
Download WSDL Flattener Tool
I’m not including source because it’s messy (I prefer to cleanup my code before sharing…) but feel free to decompile with ILSpy if you don’t trust me.

WCF is great – its flexible, standards compliant, super easy to work with, and just altogether nice. That is, until you try to use it with, say, Oracle products. WCF generates so-called modular wsdl files, with separate xsd files for types, and even sometimes multiple wsdl files. Not technically in violation of any standards, but kind-of pushing the limits.

There are plenty of articles out there about modifying a WCF service to generate a flat WSDL. Also if you upgrade to .NET 4.5, you can use the URL parameter ?singleWsdl to get a flattened WSDL.

But what if you don’t have control of the web service? What if you cannot upgrade or modify the code?

For some odd reason, Microsoft added the internal ability to generate a single WSDL but only exposed it through the WCF Service implementation. They didn’t add it to svcutil.exe, and they didn’t expose it in the Framework… publicly that is *evil grin*

Turns out, in the System.ServiceModel.Description namespace, there is an internal static class called “WsdlHelper”, and this class has a public static Method called “GetSingleWsdl” that takes a MetadataSet for a parameter, and returns a System.Web.Services.Description.ServiceDescription object representing the Wsdl as a single entity. All you have to do is call .Write(filename) and you have your file.

“But I don’t see any WsdlHelper class!!!!!!”

It’s internal – you have to use reflection to invoke it.
First, you have to load the metadata from the existing WCF service into a MetadataSet using a MetadataExchangeClient. (Hint: you will probably need to use a custom binding). Then you run that set into something like this:

            var asy = System.Reflection.Assembly.GetAssembly(typeof(WsdlExporter));
            Type t = asy.GetType("System.ServiceModel.Description.WsdlHelper", true);
            var method = t.GetMethod("GetSingleWsdl", System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Static);

            object retVal = method.Invoke(null, new object[] { metadataSet });
            //cast retVal into System.Web.Services.Description.ServiceDescription, and call the .Write() method. All done

Using this, I was able to create a tool that takes a WCF url, and allows you to save a single .wsdl file. You want the tool? Later perhaps. For now put your thinking cap and Google gloves on, and write your own.

SQL Server Configuration Manager RPC Error

On my Dev VM, I’ve been unable to manage SQL Server Express with SQL Server Configuration Manager – I get an error “The Remote Procedure call failed”.

rpc error

The solution: Updates. I found this article:

http://thesqldude.com/2012/12/05/sql-server-2012-configuration-manager-wmi-error-remote-procedure-call-failed-0x800706be/

Which mentions about things breaking wile installing Sql Server 2012 – I didn’t install that, but I did install Visual Studio 2012 – which includes Server Server 2012 components.

Install sql server 2008 sp3, and sql server 2008 r2 sp2. Problems solved.

 

Windows RT takes steps toward awesome

As I mentioned in a previous post, Windows RT is essentially Windows on ARM, and arguably that is how it should have been promoted.

Just imagine – a third platform option when downloading applications – x86, x64 and now ARM. Due to Microsoft’s arbitrary (and greed motivated) restrictions, this wasn’t supposed to happen.

Thank goodness for hacker ingenuity: not only did somebody figure out how to disable the signature requirement, somebody else created a script to simplify the unlock, and according to this article: You can jailbreak Windows RT to run desktop apps…or even Mac OS, Putty and 7-Zip have already been ported to ARM.

[Release] RT Jailbreak Tool – xda-developers

The next step is portability. If this article is to be be believed, somebody managed to get Windows RT running an an HTC HD2. If it is possible to port Windows RT to just about any ARM device… what else could it run on? Just imagine a tablet landscape where you could pick your own OS, or dual boot Windows and Android.

Imagine all the Linux nerd heads exploding if Windows RT were running on a Raspberry Pi.

Or even better – if somebody ports Windows RT to run on the iPad. THAT would be epic.

Windows RT based devices remain marginally useful and tremendously overpriced (the iPad is a good $200 cheaper… not cool). Should freedom and ingenuity prevail over Microsoft’s greed, Windows RT really could take the ARM world by storm. I’ve got my fingers crossed…

Windows 8 Store Application Development: AppBar WTF

I have a great idea for an app that I intend to submit to the Windows 8 store. I am a fairly proficient .NET developer, and I already have a nice little class library all ready to go, I just need a “Modern” UI. This shouldn’t be too hard, right?

I create my project, and start adding controls. I realize I will need that settings thing that pops up when you swipe up from the bottom of the screen. But it isn’t called “settings” – that is something different. Instead it is called the “AppBar”. Now given that I am using a tool called VISUAL Studio, the primary innovation being the ability to rapidly, visually design user interfaces, then write code for interaction. So I figure adding an AppBar should be as simple as adding a menubar was in windows forms, right?

Wrong. Dead Wrong.

An AppBar is like an iFrame. You can put whatever you want in it, and it is completely NON-obvious, and non-intuitive to do so in a meaningful fashion. The Microsoft Quick Start guide only tells you how to do it with some XAML code that they neglect to tell you where to place. Want to follow along? Here is the article: http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh781232

 

Continue reading

There is no excuse for losing your place anymore

When Compact Disks were introduced as a replacement for audio cassettes, there really was no comparison. As opposed to the analog warmth and hearth-like crackle of vinyl, nobody really misses the hissy, tinny, low-fidelity of cassettes. Similarly, few people are nostalgic for having their movie eaten by a VCR, or listening to garbled audio from a VHS tape played too many times.

There is one thing they do miss, however. When you stopped an audio or VHS tape, you knew with complete certainty that when you hit play again, it would continue where you left off. Even if you ejected the tape, and put it in a different player a year later, your position is maintained as a physical property of the media.

Optical disks have no such property – they are stateless by nature. The burden of position must be maintained by the player itself.

Back to the release of CDs and DVDs. When they came out, memory was expensive, and non-volatile memory (aka flash) prohibitively so. To replicate the saved-place of tapes, devices would need to be able to identify the disc, and have an associated position. Not easy in the 90s, so it was excusable. And honestly, losing your place in an album of music is not the end of the world. But for audiobooks it was a different story. Audiobook cassettes persisted – and arguably remain more popular that CDs to this day. The reason – losing your place in an audiobook is a HUGE pain – the sort that you never want to experience twice.

We have been experiencing a similar technology shift in recent years – from the dominance of physical media to digital media – mp3s, video files, e-books, etc. These new formats are typically stored on flash memory, so any device that can play them can store information about them… But in so many cases they do not.

Audible (arguably the world’s biggest distributor of digital audiobooks) took years to realize they should permit for synchronization of position across devices – and the last time I used it it was still unreliable. If your app crashed, you would be back to where you had been hours ago.
Amazon’s Kindle apps similarly have no excuse for losing your place in a book, and yet from time to time they do. The superior GoodReader app very easily loses position information, and it isn’t clear how to prevent it.
The Windows 8 Video player app – (which on Windows RT is your only option) – loses your current place in a movie if you close the app. (Luckily, Apple had already figured this out and has been saving your place in movies on the ipad for years).

The fact that maintaining and persisting position information is so often neglected and poorly implemented is a little disturbing – it’s simple enough to implement, it is a small quantity of information, and its necessity is obvious. I’m sure Microsoft will get around to saving position info, but why didn’t they do it in the first place? There is no excuse anymore.

I consider the following items to be self evident, necessary, and painfully obvious. Further, their omission is a negligent as omitting the “next” button.

  • Any device or software that can play audiobooks must not ever under any circumstances lose the current position of playback, and any operation that might delete said position must explicitly inform the user to that effect.
  • Any device or software that can display eBooks must remember the last page displayed for each book; must protect this information and persist it through updates, crashes and reinstalls. Any operation that might delete this information must explicitly request permission to do so.
    Additionally, bookmarks must be very easy to create.
  • Devices or software intended for watching movies must remember where you left off for recently watched movies – through closes, ejections, reboots, crashes, etc.

CRM 2011 Runtime calculated fields –retrieval behavior notes

In previous versions of CRM, creating realtime calculated fields involved a plugin that listened to the global Execute message (and optionally RetrieveMultiple if you wanted to handle filter criteria) – to handle FetchXml requests. The remaining messages (Retrieve, RetrieveMultiple, Update, Create) can be registered for the specific Entity, so you are not gumming up CRM with global plugins.

Some of the neat new features of CRM 2011 greatly complicate the process of creating a calculated field – your Plugin now must be registered globally for Retrieve, RetrieveMultiple and Execute; and there are three different query types to handle.

These are my research notes from building a plugin that allows for functionality similar to the “Customer” lookup in the built-in entities. It would be easy enough to sit on the Post-Stage event, look through the Result and use a retrieve for each record to get the calculated value. This would quickly degrade system performance – instead my solution modifies the query in the Pre-stage to include all necessary columns, then uses the data in the result to update the calculated field. It requires significantly more code, but is far more scalable.

1. Execute Message

I haven’t yet determined all the cases that Execute is called, but it is for sure called when the legacy webservice is used to perform an ExecuteFetch (such as when using Stunnware tools). It is also fired now and again by the UI (but not for normal grids – almost everything uses RetrieveMultiple now)

In the Pre-Stage, you must look for the “FetchXml” value in the InputParameters. If it is there, use your Xml parser / modifier of choice to look through it for any instances of your entity, then for each entity instance, check for your calculated field (these can be nested arbitrarily deep). If it contains the field, add your required source columns to Xml, and put the FetchXml back into the InputParameters.

The ResultXml on the Post-Stage is flat, and columns from Linked Entities are given a prefix. The prefix comes from the link-entity – it is the alias if one is specified, or the “to” attribute. (Caveat: If aliases are not used in a fetch, and there are multiple link-entities off attributes with the same name, the result will have duplicate column names. The UI always adds aliases.)

2. RetrieveMultiple Message

RetrieveMultiple is powered by a Query, and that query can be one of QueryExpression, QueryByAttribute, or FetchExpression. The query is in the InputParameter[“Query”], and is of type QueryBase.

QueryExpression

In the pre-stage, check that query.EntityName is the correct entity; check query.ColumnSet for your calculated field, adding any required fields to the ColumnSet.
Next, recurse through the LinkEntities, checking the LinkToEntityName and the Columns for each.

In the post-stage, the result is in OutputParameters[“BusinessEntityCollection”] which is of type EntityCollection. Each record is an Entity, and all fields from linked entities are in the Attributes collection (similar to Fetch), prefixed for links. The prefix is either the alias of the LinkEntity, or if the alias is blank it is the entityname followed by a sequence number (eg “contact1.fullname”, “contact1.firstname” “account2.name”. The sequence number is assigned to each LinkEntity based on the order it occurs in the LinkEntity Tree.

(Note: to be complete, something similar should be done for the Orders, so that you can sort by your calculated column)

  • Account (no prefix)
    • Contact (contact1.<attribute>)
    • Contact [alias shipper] (shipper.<attribute>)
    • User (systemuser3.<attribute>)
      • BusinessUnit (businessunit4.<attribute>)

This is better and worse than the old fetch method. It guarantees that prefixes are unique, but it requires you to traverse the entire LinkEntity tree in order to know what the resulting column name will be.

In your result, you simply do Entity[“calculatedFieldName”] = Entity[“sourceFieldName”].

 

QueryByAttribute

The simplest of all – there is no nesting, no links, no related fields. You only have to check the ColumnSet (and optionally Orders).

FetchExpression

All the same complexity as the QueryExpression, except you have to process FetchXml in the Pre-Stage. The Result works the same as with QueryExpression.

 

3. Retrieve Message

This is by far the most complicated message thanks to the RelatedEntitiesQuery property of the RetrieveRequest. I haven’t determined if / when this is used by the UI. In most cases it will be null, and you only need to worry about InputParameters[“ColumnSet”]. If InputParameters[“RelatedEntitiesQuery”] is populated, it is a keyed value collection keyed on a “Relationship” – the root of a related entity query must be part of an existing relationship. However, each query in the bag can have LinkEntities – so essentially the RelatedEntitiesQuery is a collection of multiple RetrieveMultiples attached to the record. It is a mess.

In the output, the returned Entity contains a field called RelatedEntities – this is a collection of entities again keyed on the relationship. Each of the relationship queries must be handled separately as the column aliases are specific to the query.

 

—-

I’ve encountered several discussion and blog postings about this topic, and they are generally vague or incomplete. Suffice it to say handling all ways in which a field can be “hit” in CRM 2011 is daunting.

HTC Windows Phone 8X is a mess on T-Mobile

I purchased the HTC HD7 shortly after it came out so I develop for a phone in a language I knew. Out the gate, the WP7 phone had Bluetooth issues, but these were largely fixed with the Mango update. So despite the big crack in the screen, I was happy with my Windows 7 Phone.

Two years later, and contract renewal time arrives – along with the release of the Windows Phone 8 platform. Time to upgrade! WP8 should be at least as good as WP7, right?

… Wrong.

At present T-Mobile has two Windows Phone 8 options – the HTC 8X, and the Nokia Lumia 810. The lumia wins hands-down in most categories – it includes navigation, has expandable storage, has a replaceable battery. But it is ugly, clunky, has a low resolution screen, and poor battery life.

So if you want a “nice” phone you are left with the HTC 8X – which is good enough for Gwen Stefani, so it should be pretty good, right?
Right?

Wrong. It is plagued by bugs and software ommisions, and HTC is dragging their feet to fix.

1. There is no turn-by-turn navigation.
Microsoft removed the turn-by-turn directions / navigation feature from bing maps, with the idea that nokia would pick up the slack. Despite reading that Nokia opened up Nokia Drive to all WP8 devices, this phone doesn’t have it. There is no navigation. There are no turn-by-turn directions. You can get directions in a list, but they are not suited to use while driving. I have been thrown back into the pre-GPS days, and it isn’t pleasant…
2. Internet Sharing is buggy
This may be T-Mobile specific, but for the first week after activating the phone, Internet Sharing didn’t work, claiming I had to “upsell” to a $15/month extra tethering plan. Not true – the $35/month 5-gig plan I have includes tethering. This phone likes to err on the side of “no soup for you,” and for whatever reason assumes you are not authorized to share your internet half the time. Real annoying, and a real step backwards = the HD7 never rejected me, even using a grandfathered 2-gig plan.
3. Bluetooth is a travesty of bugs and incompatibility.
It seem that HTC has real problems with Bluetooth on Windows phones. This thing cannot maintain a reliable handsfree connection for more than maybe 15 minutes before the sound cuts out. Everything appears to still be connected and working, but there is no sound. You’ll be talking, an all of a sudden – silence. You must disable the Bluetooth to resume your conversation.
The A2DP (music) profile doesn’t do this, but the phone locks up entirely for a good minute after the music connection is established. I don’t know what it’s doing, but after connecting for music in my car, there is silence for a minute, and the phone will not respond to input (although the windows button still causes haptic feedback). When it does finally unlock, it responds to all your keypresses in one batch, which is pretty annoying in is self (for example, if you held down the home button trying to wake it up and pressed the power button, when it does wake up you will be prompted for a voice command as well as a “slide down to turn off”) The Bluetooth is practically unusable in this state.
3a. While the Bluetooth technically “can” support serial and PAN, this is not supported by the OS as must be implemented on an app-by-app basis. Meaning no Toyota Entune (without significant effort my Toyota), no Bluetooth tethering. This is ridiculous. All developers shouldn’t have to implement portions of the Bluetooth stack, it should be is a shared library… oh wait, Windows phone apps cannot include or use shared libraries.
4. Custom Roms may be impossible (This applies to Windows RT as well). Microsoft was wise to require the use of a standardized boot method – the phones use a UEFI bootloader. That is awesome. What is not awesome: Microsoft requires SecureBoot be used to require roms be signed by Microsoft or the manufacturer, and does not provide the user the ability to import trusted keys. Windows phones (and rt devices) could have been the future of mobile computing – the ability to dual boot as easily as a PC – but greed and paranoia ruined it for everybody. Now we must hope that some hacker find an exploit to SecureBoot that will allow malicious software… in addition to the freedom to run what you want. Really stupid move from Microsoft.

Some of these issues can be resolved by HTC with firmware updates – and likely will in time. Others will require Microsoft to get off the pedestal of greed – and who knows when or if this will happen.

Till then, I may be stuck sharing my internet with an android device to accomplish what I need. Or I could jailbreak an iPhone to get what I need. How long till I tire of waiting… who knows.