Dirt simple method of limiting System.Threading.Task concurrency (max concurrent threads)

Microsoft has an article about creating a Task Scheduler that limits maximum concurrency:
How to: Create a Task Scheduler That Limits the Degree of Concurrency

The provided example class is clunky and kinda hard to grasp. I’ve been searching for a simple way to create a “thread pool” with limited concurrency. I previously experimented with ConcurrentBags of BackgroundWorkers – this worked, but it was also fairy complex and cumbersome.

The solution I came up with uses a Semaphore (or rather the lighter weight SemaphoreSlim type) to manage concurrency.

Simply put – you create a semaphore with a maxCount (and initialCount) equal to the max concurrency you desire. Then when you want to fire off a Task, first call semaphore.Wait(), then call semaphore.Relase() in a ContinueWith().

Contrived example:

    public class LimitedAsync
    {
        private SemaphoreSlim _semaphore;

        public LimitedAsync(int maxConcurrency)
        {
            // Create semaphore with maxConcurrency slots
            _semaphore = new SemaphoreSlim(maxConcurrency, maxConcurrency);
        }

        public void DoSomethingAsync(string param)
        {
            //Wait for semaphore to have availablilty (blocks if semaphore is full)
            _semaphore.Wait();
            //Run DoSomething in a task, then release slot in semaphore
            //ContinueWith is called even if DoSomething faults
            Task.Factory.StartNew(() => DoSomething(param)).ContinueWith((x) => _semaphore.Release());
        }

        public void DoSomething(string param)
        {
            System.Threading.Thread.Sleep(500);
        }
    }

Generating a single flattened WSDL from an existing WCF Service

UPDATE 4/12/2013: Tool included
Download WSDL Flattener Tool
I’m not including source because it’s messy (I prefer to cleanup my code before sharing…) but feel free to decompile with ILSpy if you don’t trust me.

WCF is great – its flexible, standards compliant, super easy to work with, and just altogether nice. That is, until you try to use it with, say, Oracle products. WCF generates so-called modular wsdl files, with separate xsd files for types, and even sometimes multiple wsdl files. Not technically in violation of any standards, but kind-of pushing the limits.

There are plenty of articles out there about modifying a WCF service to generate a flat WSDL. Also if you upgrade to .NET 4.5, you can use the URL parameter ?singleWsdl to get a flattened WSDL.

But what if you don’t have control of the web service? What if you cannot upgrade or modify the code?

For some odd reason, Microsoft added the internal ability to generate a single WSDL but only exposed it through the WCF Service implementation. They didn’t add it to svcutil.exe, and they didn’t expose it in the Framework… publicly that is *evil grin*

Turns out, in the System.ServiceModel.Description namespace, there is an internal static class called “WsdlHelper”, and this class has a public static Method called “GetSingleWsdl” that takes a MetadataSet for a parameter, and returns a System.Web.Services.Description.ServiceDescription object representing the Wsdl as a single entity. All you have to do is call .Write(filename) and you have your file.

“But I don’t see any WsdlHelper class!!!!!!”

It’s internal – you have to use reflection to invoke it.
First, you have to load the metadata from the existing WCF service into a MetadataSet using a MetadataExchangeClient. (Hint: you will probably need to use a custom binding). Then you run that set into something like this:

            var asy = System.Reflection.Assembly.GetAssembly(typeof(WsdlExporter));
            Type t = asy.GetType("System.ServiceModel.Description.WsdlHelper", true);
            var method = t.GetMethod("GetSingleWsdl", System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Static);

            object retVal = method.Invoke(null, new object[] { metadataSet });
            //cast retVal into System.Web.Services.Description.ServiceDescription, and call the .Write() method. All done

Using this, I was able to create a tool that takes a WCF url, and allows you to save a single .wsdl file. You want the tool? Later perhaps. For now put your thinking cap and Google gloves on, and write your own.

SQL Server Configuration Manager RPC Error

On my Dev VM, I’ve been unable to manage SQL Server Express with SQL Server Configuration Manager – I get an error “The Remote Procedure call failed”.

rpc error

The solution: Updates. I found this article:

http://thesqldude.com/2012/12/05/sql-server-2012-configuration-manager-wmi-error-remote-procedure-call-failed-0x800706be/

Which mentions about things breaking wile installing Sql Server 2012 – I didn’t install that, but I did install Visual Studio 2012 – which includes Server Server 2012 components.

Install sql server 2008 sp3, and sql server 2008 r2 sp2. Problems solved.

 

Windows RT takes steps toward awesome

As I mentioned in a previous post, Windows RT is essentially Windows on ARM, and arguably that is how it should have been promoted.

Just imagine – a third platform option when downloading applications – x86, x64 and now ARM. Due to Microsoft’s arbitrary (and greed motivated) restrictions, this wasn’t supposed to happen.

Thank goodness for hacker ingenuity: not only did somebody figure out how to disable the signature requirement, somebody else created a script to simplify the unlock, and according to this article: You can jailbreak Windows RT to run desktop apps…or even Mac OS, Putty and 7-Zip have already been ported to ARM.

[Release] RT Jailbreak Tool – xda-developers

The next step is portability. If this article is to be be believed, somebody managed to get Windows RT running an an HTC HD2. If it is possible to port Windows RT to just about any ARM device… what else could it run on? Just imagine a tablet landscape where you could pick your own OS, or dual boot Windows and Android.

Imagine all the Linux nerd heads exploding if Windows RT were running on a Raspberry Pi.

Or even better – if somebody ports Windows RT to run on the iPad. THAT would be epic.

Windows RT based devices remain marginally useful and tremendously overpriced (the iPad is a good $200 cheaper… not cool). Should freedom and ingenuity prevail over Microsoft’s greed, Windows RT really could take the ARM world by storm. I’ve got my fingers crossed…

Windows 8 Store Application Development: AppBar WTF

I have a great idea for an app that I intend to submit to the Windows 8 store. I am a fairly proficient .NET developer, and I already have a nice little class library all ready to go, I just need a “Modern” UI. This shouldn’t be too hard, right?

I create my project, and start adding controls. I realize I will need that settings thing that pops up when you swipe up from the bottom of the screen. But it isn’t called “settings” – that is something different. Instead it is called the “AppBar”. Now given that I am using a tool called VISUAL Studio, the primary innovation being the ability to rapidly, visually design user interfaces, then write code for interaction. So I figure adding an AppBar should be as simple as adding a menubar was in windows forms, right?

Wrong. Dead Wrong.

An AppBar is like an iFrame. You can put whatever you want in it, and it is completely NON-obvious, and non-intuitive to do so in a meaningful fashion. The Microsoft Quick Start guide only tells you how to do it with some XAML code that they neglect to tell you where to place. Want to follow along? Here is the article: http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh781232

 

Continue reading

There is no excuse for losing your place anymore

When Compact Disks were introduced as a replacement for audio cassettes, there really was no comparison. As opposed to the analog warmth and hearth-like crackle of vinyl, nobody really misses the hissy, tinny, low-fidelity of cassettes. Similarly, few people are nostalgic for having their movie eaten by a VCR, or listening to garbled audio from a VHS tape played too many times.

There is one thing they do miss, however. When you stopped an audio or VHS tape, you knew with complete certainty that when you hit play again, it would continue where you left off. Even if you ejected the tape, and put it in a different player a year later, your position is maintained as a physical property of the media.

Optical disks have no such property – they are stateless by nature. The burden of position must be maintained by the player itself.

Back to the release of CDs and DVDs. When they came out, memory was expensive, and non-volatile memory (aka flash) prohibitively so. To replicate the saved-place of tapes, devices would need to be able to identify the disc, and have an associated position. Not easy in the 90s, so it was excusable. And honestly, losing your place in an album of music is not the end of the world. But for audiobooks it was a different story. Audiobook cassettes persisted – and arguably remain more popular that CDs to this day. The reason – losing your place in an audiobook is a HUGE pain – the sort that you never want to experience twice.

We have been experiencing a similar technology shift in recent years – from the dominance of physical media to digital media – mp3s, video files, e-books, etc. These new formats are typically stored on flash memory, so any device that can play them can store information about them… But in so many cases they do not.

Audible (arguably the world’s biggest distributor of digital audiobooks) took years to realize they should permit for synchronization of position across devices – and the last time I used it it was still unreliable. If your app crashed, you would be back to where you had been hours ago.
Amazon’s Kindle apps similarly have no excuse for losing your place in a book, and yet from time to time they do. The superior GoodReader app very easily loses position information, and it isn’t clear how to prevent it.
The Windows 8 Video player app – (which on Windows RT is your only option) – loses your current place in a movie if you close the app. (Luckily, Apple had already figured this out and has been saving your place in movies on the ipad for years).

The fact that maintaining and persisting position information is so often neglected and poorly implemented is a little disturbing – it’s simple enough to implement, it is a small quantity of information, and its necessity is obvious. I’m sure Microsoft will get around to saving position info, but why didn’t they do it in the first place? There is no excuse anymore.

I consider the following items to be self evident, necessary, and painfully obvious. Further, their omission is a negligent as omitting the “next” button.

  • Any device or software that can play audiobooks must not ever under any circumstances lose the current position of playback, and any operation that might delete said position must explicitly inform the user to that effect.
  • Any device or software that can display eBooks must remember the last page displayed for each book; must protect this information and persist it through updates, crashes and reinstalls. Any operation that might delete this information must explicitly request permission to do so.
    Additionally, bookmarks must be very easy to create.
  • Devices or software intended for watching movies must remember where you left off for recently watched movies – through closes, ejections, reboots, crashes, etc.

CRM 2011 Runtime calculated fields –retrieval behavior notes

In previous versions of CRM, creating realtime calculated fields involved a plugin that listened to the global Execute message (and optionally RetrieveMultiple if you wanted to handle filter criteria) – to handle FetchXml requests. The remaining messages (Retrieve, RetrieveMultiple, Update, Create) can be registered for the specific Entity, so you are not gumming up CRM with global plugins.

Some of the neat new features of CRM 2011 greatly complicate the process of creating a calculated field – your Plugin now must be registered globally for Retrieve, RetrieveMultiple and Execute; and there are three different query types to handle.

These are my research notes from building a plugin that allows for functionality similar to the “Customer” lookup in the built-in entities. It would be easy enough to sit on the Post-Stage event, look through the Result and use a retrieve for each record to get the calculated value. This would quickly degrade system performance – instead my solution modifies the query in the Pre-stage to include all necessary columns, then uses the data in the result to update the calculated field. It requires significantly more code, but is far more scalable.

1. Execute Message

I haven’t yet determined all the cases that Execute is called, but it is for sure called when the legacy webservice is used to perform an ExecuteFetch (such as when using Stunnware tools). It is also fired now and again by the UI (but not for normal grids – almost everything uses RetrieveMultiple now)

In the Pre-Stage, you must look for the “FetchXml” value in the InputParameters. If it is there, use your Xml parser / modifier of choice to look through it for any instances of your entity, then for each entity instance, check for your calculated field (these can be nested arbitrarily deep). If it contains the field, add your required source columns to Xml, and put the FetchXml back into the InputParameters.

The ResultXml on the Post-Stage is flat, and columns from Linked Entities are given a prefix. The prefix comes from the link-entity – it is the alias if one is specified, or the “to” attribute. (Caveat: If aliases are not used in a fetch, and there are multiple link-entities off attributes with the same name, the result will have duplicate column names. The UI always adds aliases.)

2. RetrieveMultiple Message

RetrieveMultiple is powered by a Query, and that query can be one of QueryExpression, QueryByAttribute, or FetchExpression. The query is in the InputParameter[“Query”], and is of type QueryBase.

QueryExpression

In the pre-stage, check that query.EntityName is the correct entity; check query.ColumnSet for your calculated field, adding any required fields to the ColumnSet.
Next, recurse through the LinkEntities, checking the LinkToEntityName and the Columns for each.

In the post-stage, the result is in OutputParameters[“BusinessEntityCollection”] which is of type EntityCollection. Each record is an Entity, and all fields from linked entities are in the Attributes collection (similar to Fetch), prefixed for links. The prefix is either the alias of the LinkEntity, or if the alias is blank it is the entityname followed by a sequence number (eg “contact1.fullname”, “contact1.firstname” “account2.name”. The sequence number is assigned to each LinkEntity based on the order it occurs in the LinkEntity Tree.

(Note: to be complete, something similar should be done for the Orders, so that you can sort by your calculated column)

  • Account (no prefix)
    • Contact (contact1.<attribute>)
    • Contact [alias shipper] (shipper.<attribute>)
    • User (systemuser3.<attribute>)
      • BusinessUnit (businessunit4.<attribute>)

This is better and worse than the old fetch method. It guarantees that prefixes are unique, but it requires you to traverse the entire LinkEntity tree in order to know what the resulting column name will be.

In your result, you simply do Entity[“calculatedFieldName”] = Entity[“sourceFieldName”].

 

QueryByAttribute

The simplest of all – there is no nesting, no links, no related fields. You only have to check the ColumnSet (and optionally Orders).

FetchExpression

All the same complexity as the QueryExpression, except you have to process FetchXml in the Pre-Stage. The Result works the same as with QueryExpression.

 

3. Retrieve Message

This is by far the most complicated message thanks to the RelatedEntitiesQuery property of the RetrieveRequest. I haven’t determined if / when this is used by the UI. In most cases it will be null, and you only need to worry about InputParameters[“ColumnSet”]. If InputParameters[“RelatedEntitiesQuery”] is populated, it is a keyed value collection keyed on a “Relationship” – the root of a related entity query must be part of an existing relationship. However, each query in the bag can have LinkEntities – so essentially the RelatedEntitiesQuery is a collection of multiple RetrieveMultiples attached to the record. It is a mess.

In the output, the returned Entity contains a field called RelatedEntities – this is a collection of entities again keyed on the relationship. Each of the relationship queries must be handled separately as the column aliases are specific to the query.

 

—-

I’ve encountered several discussion and blog postings about this topic, and they are generally vague or incomplete. Suffice it to say handling all ways in which a field can be “hit” in CRM 2011 is daunting.

HTC Windows Phone 8X is a mess on T-Mobile

I purchased the HTC HD7 shortly after it came out so I develop for a phone in a language I knew. Out the gate, the WP7 phone had Bluetooth issues, but these were largely fixed with the Mango update. So despite the big crack in the screen, I was happy with my Windows 7 Phone.

Two years later, and contract renewal time arrives – along with the release of the Windows Phone 8 platform. Time to upgrade! WP8 should be at least as good as WP7, right?

… Wrong.

At present T-Mobile has two Windows Phone 8 options – the HTC 8X, and the Nokia Lumia 810. The lumia wins hands-down in most categories – it includes navigation, has expandable storage, has a replaceable battery. But it is ugly, clunky, has a low resolution screen, and poor battery life.

So if you want a “nice” phone you are left with the HTC 8X – which is good enough for Gwen Stefani, so it should be pretty good, right?
Right?

Wrong. It is plagued by bugs and software ommisions, and HTC is dragging their feet to fix.

1. There is no turn-by-turn navigation.
Microsoft removed the turn-by-turn directions / navigation feature from bing maps, with the idea that nokia would pick up the slack. Despite reading that Nokia opened up Nokia Drive to all WP8 devices, this phone doesn’t have it. There is no navigation. There are no turn-by-turn directions. You can get directions in a list, but they are not suited to use while driving. I have been thrown back into the pre-GPS days, and it isn’t pleasant…
2. Internet Sharing is buggy
This may be T-Mobile specific, but for the first week after activating the phone, Internet Sharing didn’t work, claiming I had to “upsell” to a $15/month extra tethering plan. Not true – the $35/month 5-gig plan I have includes tethering. This phone likes to err on the side of “no soup for you,” and for whatever reason assumes you are not authorized to share your internet half the time. Real annoying, and a real step backwards = the HD7 never rejected me, even using a grandfathered 2-gig plan.
3. Bluetooth is a travesty of bugs and incompatibility.
It seem that HTC has real problems with Bluetooth on Windows phones. This thing cannot maintain a reliable handsfree connection for more than maybe 15 minutes before the sound cuts out. Everything appears to still be connected and working, but there is no sound. You’ll be talking, an all of a sudden – silence. You must disable the Bluetooth to resume your conversation.
The A2DP (music) profile doesn’t do this, but the phone locks up entirely for a good minute after the music connection is established. I don’t know what it’s doing, but after connecting for music in my car, there is silence for a minute, and the phone will not respond to input (although the windows button still causes haptic feedback). When it does finally unlock, it responds to all your keypresses in one batch, which is pretty annoying in is self (for example, if you held down the home button trying to wake it up and pressed the power button, when it does wake up you will be prompted for a voice command as well as a “slide down to turn off”) The Bluetooth is practically unusable in this state.
3a. While the Bluetooth technically “can” support serial and PAN, this is not supported by the OS as must be implemented on an app-by-app basis. Meaning no Toyota Entune (without significant effort my Toyota), no Bluetooth tethering. This is ridiculous. All developers shouldn’t have to implement portions of the Bluetooth stack, it should be is a shared library… oh wait, Windows phone apps cannot include or use shared libraries.
4. Custom Roms may be impossible (This applies to Windows RT as well). Microsoft was wise to require the use of a standardized boot method – the phones use a UEFI bootloader. That is awesome. What is not awesome: Microsoft requires SecureBoot be used to require roms be signed by Microsoft or the manufacturer, and does not provide the user the ability to import trusted keys. Windows phones (and rt devices) could have been the future of mobile computing – the ability to dual boot as easily as a PC – but greed and paranoia ruined it for everybody. Now we must hope that some hacker find an exploit to SecureBoot that will allow malicious software… in addition to the freedom to run what you want. Really stupid move from Microsoft.

Some of these issues can be resolved by HTC with firmware updates – and likely will in time. Others will require Microsoft to get off the pedestal of greed – and who knows when or if this will happen.

Till then, I may be stuck sharing my internet with an android device to accomplish what I need. Or I could jailbreak an iPhone to get what I need. How long till I tire of waiting… who knows.

Dynamics CRM 4.0 truly calculated fields

Microsoft has a real gem with Dynamics CRM. It is both an application and a platform, highly customizable without writing a line of code, and once you throw in the code you can do most anything (depending on how unsupported you want to go)

Internally, CRM has a number of calculated fields. The customer lookup, for example is defined in the database as COALESCE(accountid, contactid). If you wanted to be really bold, I supposed you could create a column then modify it to be a function within the database, but that would be an extremely bad idea. The correct way is to use plugins and / or javascript.

Form JavaScript is fine when you are only considered with maintaining the value when a use opens the form, but what if they print? Or use the field in a view? Or another application uses the web services to retrieve the value?

You need a plugin, and that plugin has a lot of work to do.

I recently created a plugin that permits the creation of lookups that target multiple entities. After finally handling (or chosing not to handle) all the contingent scenarios I learned a few things.

1. Your plugin has to handle the Execute message, as this is the message used for FetchXml. You will have to parse the fetchxml, determine if your entity is in it, determine if your column is in it (and you have to recurse through the link-entities).
You can either register on the post stage and perform your own retrievals to get the data for the calculation, or you can sit on the pre and post stages and modify the fetchxml in the pre stage to include the extra columns needed for your calculated field. This is MUCH MUCH faster. You will need to determine what the ResultXml looks like based on how the fetch is designed.
2. Despite what the Microsoft Documentation suggests, you do not have to register the ReteieveMultiple message for all entities – just the entity in question.
(Unless you intend to handle your calculated field being used in a filter condition. This could get extremely complicated and it is better to just make the field non-searchable)
3. You should make sure your calculated field is configured as non-searchable. This prevents it being used as a filter condition from the UI (advanced find and the like)
4. The same performance recommendation holds for the Retrieve message – register pre and post, modify the ColumnSet to contain the fields needed for the calculation, then modify the result set.
5. CRM doesn’t care if extra columns show up in the result set.
6. If your field is editable (the calculation is two-way) you will need to handle the create and update, filtered for your calculated field. Add the changes to the underlying fields to the InputParameters Target propertybag.

A couple notes about CRM 2011:
I haven’t found explicit documentation of this fact, but Fetch requests no longer use the Execute message – they are instead RetrieveMultiple.
There are two reasons I have inferred for this:
1: It is now possible because the QueryExpression in CRM 2011 allows LinkEntities to have a ColumnSet – thus providing feature parity between Queries and Fetch.
2: Plugins generally now execute within the CRM transaction, so most likely the FetchXml is parsed into a QueryExpression before the plugin is fired.
This is good and bad – it reduces the number of messages you have to listen for, but it means you do have to hook the global RetrieveMultiple.

Since CRM 4.0 is pretty dated I’m guessing there isn’t too much demand for a multilookup akin to the Customer lookups – but I have one, and I have managed to retool the basic functionality to work in CRM 2011 as well. (Who knows, it may end up as a solution!)

Windows 8 RT feels like Windows on ARM. Microsoft would be wise to embrace that.

I just got the chance to play with a Surface RT tablet, and I’m actually feeling much more positive about it. My understanding was that RT devices would pretty much only run Windows-Store, aka “Modern”, aka Metro apps, with cursory access to the desktop.

Turns out, Windows RT CAN run desktop apps. It has notepad, it has calculator, it has the command prompt and the administrative command prompt. Office 2013 is a desktop app on RT. It would appear that the infrastructure for windows desktop applications is alive and well on Windows RT, but Microsoft has taken steps to prevent us (us being developers) from using it.

This will inevitably change. Whether Microsoft condones it or not, people will figure out how to compile and load windows apps on ARM. (I’m REALLY hoping Microsoft provides a permitted way of doing it…, but I’ll take a jailbreak if necessary)

 

When Microsoft allows people to freely develop and install apps of all kinds on their Windows 8 devices (ARM or x86), Windows may just reach the holy grail of platform independence… on their flagship desktop operating system.

 

I’m really hoping they realize how huge this could be – sure they make a lot of money taking a 30% cut from the Microsoft Store, but restrictions and fees will only slow platform adoption. Restricted, closed platforms is the norm on tablets. An open, familiar platform… THAT would be game changing.

 

Keeping my fingers crossed…