A recurring theme in web programming is calling a function periodically and/or at a specific date and time. This has two aspects:

• Calling a function on a scheduled basis
• Making sure time-outs don't interfere

Calling a function on a scheduled basis
To be able to call a function on your web app, you first need an endpoint (a URL) that you can call to kick the function off. In ASP.NET you can do this in several ways:

1. Create a page that calls the function.
2. Create a handler (ASHX) that calls the function (more efficient than a page).
3. Create a WCF service that allows calls with HTTP GET, as discussed in this blog post by Sasi Suryadevara.

With your endpoint in place, you can use the Windows Task Scheduler to invoke the function at any given time and at intervals as low as one minute. With the Windows Task Scheduler you have several options again:

Create a VB Script that calls the URL, as discussed in this blog post by Steve Schofield.

Create a PowerShell script that calls the URL (same as option 1, but more modern).

Have the Windows Task Scheduler open Internet Explorer and open the specified URL (e.g. C:\PROGRA~1\INTERN~1\iexplore.exe  -extoff http://www.google.com, which starts IE without extensions). If you do this, you also need to specify that the Task Scheduler closes IE after 1 minute, which you can do in the Settings tab of the task (Windows 2003), or in the Trigger configuration (Windows 2008), as shown below.

Task Settings in Windows 2003

Trigger configuration in Windows 2008

Note: In Windows 2008 the dropdowns governing duration and interval show 30 minutes as lowest value. You can in fact change this to 1 minute by editing the text.

Making sure time-outs don't interfere
A web based call is bound to time-out after a few minutes. If you task takes longer than that, this may abort the call depending on how you programmed it, and what webserver settings are used with regards to disconnected clients. To ensure a time-out does not interfere, you can spawn a new thread and have it call the function. That way the thread handling the request can return a response to the client, and the function is carried out regardless. One issue that may arise there is that the function itself hangs or takes too long. You may want to add logic to ensure that it's aborted after a certain time, and add logging to notify you of this, and possibly also ensure that the function can only be run by one caller at a time.

Sometimes we come across integration scenario's that look straighforward, but where the devil is in the details. We needed to integrate our asp.net/silverlight application in an existing ASP "classic" site (yes, the still exist). The catch was that we needed to call the ASP "classic" site in a server to server call to get some information, but we needed to do this under the context of the current user. You may be wondering why we didn't go through a shared database or someting, but the problem is that there is little knowledge left of the old app, so changing the existing app was a no go.

So, in order to impersonate the user, you need your server-sided request look like that user. This means forwarding the cookies the user sends, and sending back the cookies the server sends to the user. Below is code that demonstrates that.

          HttpWebRequest
webRequestToServer = (HttpWebRequest)HttpWebRequest.Create("http://somedomain/somepage.asp");
webRequestToServer.CookieContainer =new CookieContainer(); foreach (String
cookieKey in Request.Cookies)
{ HttpCookie cookie = Request.Cookies[cookieKey];
Cookie serverCookie =new Cookie(cookie.Name,
cookie.Value, "/", "somedomain");
webRequestToServer.CookieContainer.Add(serverCookie); } HttpWebResponse webResponseFromServer = (HttpWebResponse)webRequestToServer.GetResponse(); foreach (Cookie
serverCookie in webResponseFromServer.Cookies)
{ HttpCookie clientCookie = Response.Cookies[serverCookie.Name]; if (clientCookie
== null)
{ clientCookie =new HttpCookie(serverCookie.Name);
} clientCookie.Value = serverCookie.Value;
clientCookie.Expires = serverCookie.Expires;
Response.Cookies.Add(clientCookie); } webResponseFromServer.Close();
        

This code works fine in a test environment, but there is a catch... in some cases the domain of the server is not set in the cookie you get on the server side. The problem with that is that when you set the domain, it doesn't correspond to what the server expects. You can see this if you write out the cookies you send/receive (both on the browser connection and te server-server connection) to a log or something (including the domain. It took a while to figure out, but replacing "somedomain" with Request.ServerVariables["LOCAL_ADDR"] did the trick.

When you're using signing or encryption on your SOAP requests, WCF exepects the response to be signed/encrypted too. When the response is not signed/encrypted the message encoder throws a MessageSecurityException. This is perfectly fine behavior, but in interop scenario's can really bug you, because some WS-* implementations don't sign/encrypt Fault messages. Now, because the message encoder throws the exception, you can't get to the underlying SOAP fault. This means that you have no clue why you received a fault in the first place.

To fix this, Microsoft has provided a hotfix. With this hotfix in place you can specify enableUnsecuredResponse="true" in the binding configuration to allow unsecured responses. Unfortunately this means that also valid responses don't have to be signed/encrypted, defeating the purpose of signing and encryption altogether!

As an alternative, you can implement your own message encoder that wraps the encoder that is actually used. In the wrapper you can either store the received XML for use higher up in the call stack, or retrieve the fault and throw a FaultException<>. Without jumping through hoops the latter option does require your wrapper to know about the fault types it needs to handle. With the former option you can handle the exception higher up in the call stack by catching the MessageSecurityException and throwing a new exception with the XML of the message as a property.

If you've ever tried svcutil.exe to import WSDL which has doesn't have <sp:OnlySignEntireHeadersAndBody> specified in the security policy, you'll know that this doens't fly. SvcUtil will tell you the the security policy is not supported. So why is this? I assume this has something to do with the a statement in paragraph 6.6 in the WS-SecurityPolicy specification, which states:

Setting the value of this property to 'true' mitigates against some possible re-writing attacks.

So apparently Microsoft decided that setting it to false is not a good idea, and decided not to support setting it to false (omitting the element).

Talking to a non-WCF webservice is like a box of chocolates... you never know what you're going to get. After solving the issue mentioned in my previous blog post, I had another problem. For some reason the service didn't expect a <wsa:ReplyTo> element if the value was anonymous. Later on the other party adjusted the service so it actually worked as expected from WCF, but in the mean time I did write a message inspector to solve the problem. Besides solving the problem it also is a nice little example of a message inspector.

          
            public
            class RemoveAnonymousReplyToMessageInspector
: IClientMessageInspector { privateconststring ReplyToNode ="ReplyTo"; privateconststring WSAddressingNamespace ="http://www.w3.org/2005/08/addressing"; publicvoid AfterReceiveReply(ref System.ServiceModel.Channels.Message
reply, object correlationState)
{} //
Not used for this scenario.publicobject BeforeSendRequest(ref System.ServiceModel.Channels.Message
request, System.ServiceModel.IClientChannel channel) { //
This method is called before the request is sent. You can read/manipulate the message
here. // If you're using signing or encryption, that is done after this, this is the
// unencrypted/unsigned mesage. request = RemoveAnonymousReplyTo(request); returnnull;
} private Message
RemoveAnonymousReplyTo(Message message) { if (message.Headers.ReplyTo.IsAnonymous
== true)
{ int index = message.Headers.FindHeader(ReplyToNode,
WSAddressingNamespace); message.Headers.RemoveAt(index); } return message;
} } 
        

To use this, you'll need to create a class implementing the IEndpoint behavior and add the MessageInspector in ApplyClientBehavior, as follows:

          
            public
            void ApplyClientBehavior(ServiceEndpoint
endpoint, ClientRuntime clientRuntime) { RemoveAnonymousReplyToMessageInspector inspector =new RemoveAnonymousReplyToMessageInspector();
clientRuntime.MessageInspectors.Add(inspector); }
        

Recently I found myself trying to talk to a webservice using signing. It was a WCF calling a Java webservice using a certificate to sign messages. I kept getting the following exception message:

The incoming message was signed with a token which was different from what used to encrypt the body. This was not expected.

After a wild goose chase we finally figured out that the certificate was corrupted. Just installing the certificate again solved the issue.

I've written quite a few functional designs over the years and I've found that for users needing to validate it having visuals is key. In most cases prospective users don't understand what they really get until they see screens. On the opposite side of the spectrum telling developers what to do also is much easier with a screen, especially when you are debating what would be the best and most efficient (coding wise) way to give a user certain functionality. In these situations just getting a piece of paper and draw is the best you can do. The last few years I've done this on and off on my tablet. I can sketch on it, but the results are often so poor to see (and read!), that I can't possible put it in a functional design. This is where a good mockup tool comes in.

A good mockup tool should make you feel like you are drawing, but provide you with predefined controls to make your job fast and easy. Recently I came across Balsamiq Mockups, which is simply jaw dropping. Let's start with the result, which looks pretty much like a hand drawn thing. At first glance that may not seem like a big deal, but it is. It states clearly "This is a mockup, the actual thing may look different". If you give a user something that looks like a screen shot, that is what to expect to get. With this they know it will look differently when it is done, and this also makes it much easier to debate your choices and come up with better ideas (to quote David Platt, "Thy User Is Not You", so they will come up with stuff you didn't even dream about).

Ok, so the result is great, what about getting there? Well, that's a piece of cake, really. Balsamiq is as intuitive a tool as I've seen and I was able to create a pretty complex screen in about 10 minutes. There's a bunch of commonly used controls (and some less common), and you can easily find what you need. Also, you can download tons of additional controls from http://mockupstogo.net. Placing, moving, resizing etc. is all very easy because of the snapping support. Want to see for yourself? Look here.

The last things that I find refreshing is the licensing model and fee. It only costs $79 for a single license, and that comes with updates forever (and they update frequently, so they say). Because the tool is already so good, this means you can use it for years, without having to worry about support or having to get a new version.

This is just a great tool. I am sure I will be using it often.

Recently we moved our projects to a new TFS server. This means that all workspaces link to the old TFS, and you will get the error message "The path [path] is already mapped in workspace [workspace]" when you try to add a new reference to a project you've already worked on. I keep running into this every once in a while, because we develop from virtual machines and we have different ones for different projects. I also keep forgetting what to do, so this is partly a message to self.

You can solve this by opening the Visual Studio command prompt and typing:

tf workspaces /remove:[projectname]

If you want to remove all of them (which is the case when switching to a new TFS), you can use a wildcard, like this:

tf workspaces /remove:*

Thanks to Gergely Orosz's post on the topic.

I've been reading parts of this book on and off for a long time, and I've pretty much read the whole book and some sections more than once, so it is time for a review (even though this is an "older" book).

If you're a software developer, this is a must read. It gives you a kick in the head about what you should think of when you develop software. The examples David Platt uses in this book are very striking, and will make you think about what you've built yourself. You may feel embarrased sometimes and think "I did that too, shame on me!" However, in the end it will make you a better developer. Platt shows very well that IT is no longer about technology, but about the people operating the technology.

This book is also a fun read. Platt has a good sense of humor, and in some cases the stories he tells themselves are pretty hilarious. The only downside of the book is that at some point repetition kicks in. The message is clear, so you want to move on, but there are still more examples explaining the same thing.

We're doing a project where were talking to non-.NET web services and this means that sometimes we have to see what the exact message looks like. In case you ever need to do this: a MessageInspector will not (always) be suitable for this. This is particularly the case where you're using a certificate to sign the message. Signing happens after the MessageInspector is invoked, so it will just show you the unsigned message. One thing you can do is setup a URL you can post to that logs the incoming message as is. We have a "service" like that available to all developers, because it is a quick and easy solution that doesn't require installing additional tooling or modify code. Another option is to configure message logging (see the MSDN article Recommended Settings for Tracing and Message Logging).

At my company we we're looking at creating a generic STS that does not require Active Directory Federation Services 2.0, and we were also thinking about putting it up on CodePlex. Dominick Baier from Thinktecture beat us to it with StarterSTS. He's also posted some webcasts on how to use it. Good stuff, so instead rolling our own, we'll be using/extending this one.

Windows Identity Foundation introduces a new ClaimTypes class. It contains predefined claim type URIs for claims defined by OASIS and Microsoft. In the WIF SDK project templates for a custom STS this ClaimTypes class is mixed with the one already in System.IdentityModel.Claims, which is rather confusing. So, what's the difference?

Functionally: None. All claim type URIs in Microsoft.IdentityModel.Claims.ClaimTypes are identical to corresponding types in System.IdentityModel.Claims.ClaimTypes. That said, Microsoft.IdentityModel.Claims.ClaimTypes adds a few new claim types.

Technically: Claim types in System.IdentityModel.Claims.ClaimTypes are defined as static read only string properties, whereas in Microsoft.IdentityModel.Claims.ClaimTypes the claim types are string constants.

My advice: for clarity always use Microsoft.IdentityModel.Claims.ClaimTypes.

As many BizTalk developers we've been using the SSO (Single Sign On) database for more than just SSO. We've been storing configuration data in that database too. This used to be a work around, and we were using some custom tool to edit the entries in the SSO database. However, we recently learned that Microsoft has release an MMC snap-in to manage the values. You can download it here. Microsoft itself states this about the tool:

This tool provides the ability to add and manage applications, add and manage key value pairs, as well as import and export configuration applications so that they can be deployed to different environments.

That apparently means that even Redmond has "officially" accepted that SSO is used for configuration as well.

Recently I found myself having to debug a large stored procedure that calls all sorts of sub-procedures. Also, the data set was rather large and it only failed sometimes. SQL Management Studio in this scenario isn't very helpful for several reasons:

• The number of PRINT messages that it'll show is limited, so if the error is out of range... tough luck.
• Even with PRINT messages it can be hard to pin down which statement is actually in error.
• Line numbers are never accurate.
• It seems View Dendencies sometimes misses out on dependencies.
• View Dependencies doesn't show the number of times a proc is called from another proc.

SQL Profiler, which also comes with SQL Server is a much better tool when it comes to pin-pointing the erronous statement(s), after which you can modify your code to show an error message that prints the values causing the hickup. When you use SQL Profiler, be sure to select more than just the default events. Click Show All Events and select all Error events, as well as the Stored Procedure events that show Starting and Completed of SQL statements with stored procedures. Then, once the root procedure is done, do a Find through the trace for "Error", and you can see exactly which statement is causing the error.

My company regularly works on public facing websites, and as such it is imperative we test the sites we create with most common browsers. Naturally that means at least Internet Explorer, FireFox, Safari, and Opera. With the last three we just download and install om some test (virtual) machine. With IE however this is somewhat more complicated (although not impossible to run different versions of IE side by side. However, Microsoft provides a set of Virtual PC images known as the Internet Explorer Application Compatibility VPC Image. These images enable you to test different versions of IE on different versions of Windows. These images have a limited lifetime (between 1-4 months), so you'll have to download a new set on a regular basis, but other than that this is really handy. The following configurations are available:

• Windows XP SP3 with IE6
• Windows XP SP3 with IE7
• Windows XP SP3 with IE8
• Windows Vista with IE7
• Windows Vista with IE8

Unfortunately these configurations are all en-US, so if you want to test with say a Dutch version of Windows, you'll have to create your own images (which is what my company has done, even for en-US).

Today http://orchard.codeplex.com/ went live. Orchard is an open source Content Management System that the folks from Redmond are working on together with the ASP.NET community, and which I've been following with much interest. Orchard is based on ASP.NET MVC, which means I'd favor it over something like Umbraco when it is mature enough. Why? Because this should mean that it blends easier with your regular development efforts, rather than having to deal with an entirely different templating technology. Umbraco for instance uses XSLT, and even though I wrote a book a about XSLT and my company is well versed in XSLT because we do a lot of BizTalk, it is troublesome for plain ASP.NET developers.

Keep in mind that Orchard is relatively new and a lot of scenario's are still not supported. But at the pace the team is going, you'll soon see more advanced stuff being possible. You can make yourself heard about what you'd like to see through CodePlex or through one of the sessions at TechEd or PDC this month.

Lately I've been making time to read more again, and I thought I'd share my findings. My most recent read is Solid Code: Optimizing the Software Development Life Cycle by Donis Marshall and John Bruno. My feelings about this book are mixed. On the one hand, it gives a good overview of software engineering practices. On the other hand, an overview is all that it is. It doesn't really do a good job at giving any details. That said, I think it is a good read for junior/medior developers to get a sense of all the stuff they should be aware of when building software. For senior developers and architects a quick scan to see if they have a gap somewhere is enough. From there you can explore books specialized at the topic(s) you need to know more about. Some of the topics covered include, design, testing, performance, scalabiltity, and security.

As an aside, I've read several books now that cover Agile development methodologies. What strikes me everytime is that the given examples are always about product development, and this book is no different. Product development and custom software projects however are very different. In product development you can work with fixed budget and fixed time, and cut features if either budget or time doesn't allow you to create them. Custom software projects can't do this as easily. Clients are not going to give you a bag of money and a schedule and say "we'll see what makes it into the final version". The client wants to know what he/she is going to get for the money they pay. This means that in custom software projects, you need to have a much more detailed view up front of what needs to be implemented. I'm not saying this precludes Agile development practices, but there is a certainly a difference at the start of the project, because a large chunk of the design work has to be done earlier in the project lifecycle.

Every once in a while (when I use a new dev environment) I hit this error:

Saving Changes in not permitted. The changes you have made require the following tables to be dropped and re-created. You have either made changes to a table that can’t be re-created or enabled the option Prevent saving changes that require the table to be re-created

Each time I forget how to fix this, so by posting here I know I'll never forget. Check Pinal Dave's blog post on this error (hint: it's in the Option menu). Thanks Pinal!

I was working on this little app I wrote a while ago and wanted to add some features requiring (de)serialization. So, I took the original class and made it a DataContract so I could use it with the DataContractSerializer. The class then looked more or less like this:

          [DataContract] publicclass MyClass
{ List<Trip> m_Items =new List<Item>();
[DataMember] public IList<Item>
Items { get { return m_Items;
} } }
        

Serialization went fine, but when I tried to deserialize the same object, I got a null reference exception. Of course you say, you should have added a method tied to de OnDeserializing event, because the constructor of the object doesn't work and hence the m_Items field is never initialized. The code I added to solve this looked like this:

          [OnDeserializing] protectedvoid Init(StreamingContext
context) { m_Items =new List<Item>();
}
        

To my surprise I still got the same exception. I finally figured out that the problem was the type of Items. It is was an IList<> instead of a List<>. To avoid tying a class to a specific implementation of a list, I usually use an interface, which is good practice in most cases... however, not when you want to do deserialization :).

At BataviaLabs we were debating coding guidelines the other day and came across this one: do you use #if or the ConditionalAttribute to indicate to the compiler if a method should be compiled. Let me elaborate...

If you have a method you only want to compile in a debug scenario, you have the following options:

1) Use #if DEBUG as shown below

          
            class Program
{ staticvoid Main(string[]
args) { #if DEBUG
SomeMethod(); #endif Console.WriteLine("End");
Console.ReadKey(); } #if DEBUG internalstaticvoid SomeMethod()
{ Console.WriteLine("SomeMethod");
} #endif }
        

2) Use the ConditionalAttribute as shown below

          
            class Program
{ staticvoid Main(string[]
args) { SomeMethod(); Console.WriteLine("End");
Console.ReadKey(); } [Conditional("DEBUG")] internalstaticvoid SomeMethod()
{ Console.WriteLine("SomeMethod");
} }
        

The difference between these two methods is enormous. The first sample is very explicit. Any code you don't want to compile into the production build is placed between #if DEBUG and #endif. If you try to call SomeMethod in a production build, the compiler will give you a compile error. The ConditionalAttributeon the other hand doesn't require you to remove the calls to SomeMethod. If a method is marked [Conditional], any calls made to that method are removed from the build by the compiler. A proviso here is that [Conditional] only works with methods that don't return a value (i.e. void).

I much more prefer #if DEBUG, because it is explicit. I can't run into a situation where from reading the code I'm thinking "SomeMethod is being executed", but it actually isn't because the compiler removed the call. Comments anyone?

When testing a UI, especially a web UI, it is imperative you do so at all resolutions you expect your users to use. So, how do you size the browser to 1024x768 on a 1680x1050 screen? The answer is a little tool called Sizer. It allows you to set a window to a prefixed size or you can drag the window size and it will show the actual size as a tooltip.

I read this post from Steven Martin at Microsoft and frankly I'm disappointed. Microsoft is not the only company building cloud computing services, but they have a clear advantage over most of the providers: they own the operating system. As such, a unique selling point would definitly be that they can provide you with cloud services, but also enable you to run your applications in your own data center without modifications. If I build an application for the Windows platform, I want to build it once and be able to run in on any server infrastructure. As it looks now, this is not possible. Once built for the cloud, it must remain in the cloud unless you refactor the application for use in your own environment. I really hope Microsoft sees that this is a mistake and that it will actually gain them clients if they allow this. There is another factor here and that's trust. I'd like to have a backup scenario in case Microsoft fails to deliver. With the Azure platform as is, there is no backup scenario. You either go for it full-blown, or you don't. It is my belief that many people will decide not to go with Azure in the first place because of this. In fact, I am now much more reluctant to tell my clients about Azure as an option.

Recently my latest (Dutch) book was released, titled XML - de basis. For that book I did an interview with Chris Lovett from the Microsoft XML Team. Below is the unedited version of that interview. There's a lot of cool stuff in the interview which I unfortunately had to cut for the book, because otherwise it wouldn't fit the number of pages available for the interview. Fortunately, my blog doesn't have that limitation :).

Please tell us who you are and what you do.
My name is Chris Lovett, and I’m an architect on the Data Programmability Tools team in SQL Server, and I work on XML tools that ship in Visual Studio. As an architect I do lots of different things including: hands on product development on the XML editor in Visual Studio 2008 “Orcas”; cross-group collaboration to make sure all our tools fit together; playing with other concepts like XML Notepad; and thinking about and communicating our strategy and future directions for our technology.

How long have you worked for Microsoft and what did you do before your current position?
I have worked on pretty much every XML core technology from Microsoft starting with MSXML in 1997, then to System.Xml in the .NET frameworks v1, and 2 and for the last few years I’ve been focusing on building XML tools in Visual Studio – for example, I was the primary developer on the XML editor in Visual Studio 2005. Before Microsoft I worked at IBM on OS/2 applications, then I joined the IBM Apple/IBM joint venture called Taligent, then I started my own company in Sillicon Valley with a bunch of friends from Taligent during the height of the .com boom and that’s what led me to Microsoft. A lot of the .NET Framework works with XML in some form or another.

Why is XML such a key component and what were the challenges you faced because of that?
For one thing the .NET frameworks were designed during the peak of the XML hype curve J, but more seriously, many folks at Microsoft were waking up to the fact that not everything had to be buried in code. Some things could be very neatly described at a higher “declarative” level and HTML showed the world that “markup” is a great way to do that. So XML became the way to specify configuration information (.config files), and build information (MSBuild files), and setup information (WIX), object remoting with SOAP, and security permissions and so on. All of these domain specific uses of XML were then supported by our core System.Xml classes. It’s interesting to note that even those teams that didn’t swallow XML back in .NET v1.0 are fixing that, for example, we have the new AJAX work from the ASP.NET team and we have XAML in the Windows Presentation Framework (I’m a huge fan of WPF by the way. I’ve done a lot of UI development in the last 20 years and I have to say WPF totally rocks). So XML is touching everything from database, management, communications, content publishing and now even into the user interface layer. I was amazed at the last PDC just about every talk showed some snippet of XML somewhere during their talk.

I remember when we started XML at Microsoft most people thought we were crazy. The biggest challenge was convincing people the cost of parsing and storing verbose XML tags was worth it and our team has been working on performance, and scalability ever since. But the technical challenges are easy to overcome. The real reason XML become a key component of .NET, (and Windows, and SQL Server and Office) is because it achieved true cross-platform interoperability and because it was good enough for that job, which leads to the next question…

What do you (personally) like the most about XML and its associated standards?
Simplicity, cross-platform interoperability and huge adoption. The great thing about XML is that it is humble. It’s not trying to solve world hunger. Just invent your own tags, group them into structures that make sense for your domain and viola. It is very simple and it is this concept that helped HTML take the world by storm. XML then improved on HTML by providing a clean separation between data and UI. It brought MVC to the masses so to speak.

The other day my 12 year old son was all excited and just had to show me what he discovered. He was editing Age of Empires XML files to tweak the behavior of the game using Notepad. I asked him how he knew that he could do that and how he knew the XML syntax. He didn’t know what “syntax” meant, but he knew how to edit XML ! Then the same week my doctor was all excited when he heard I worked on XML because he was involved in a software purchasing decision at our local hospital and it came down to their level of XML support. I couldn’t believe my ears.

The funny thing is that most programmers don’t really like XML. Probably because it doesn’t use curly brackets J, so most programmers treat XML a bit like the ugly duckling. But the reality is that the whole world gets markup – to them markup makes our programming world more approachable. There are still way too many programmers that don’t get this.

What would be your #1 tip to people learning XML, XML Schema, XSLT and XQuery?
First of all I would say that XML 1.0 is the foundation. A must have. Can’t go wrong there, learn it, and learn how XML encoding works – that’s the number one issue people have with XML 1.0 – people don’t take the time to understand how UTF-8 encoding works which is a pretty important foundation to XML. Don’t worry too much about DTD, because we now have XML schema.

XML Schema (XSD) is a layer on top of XML 1.0. When you need a way to describe your XML structure in “standard contract” XSD is handy and most importantly – it’s there and it’s a standard. So don’t re-invent the wheel, but I’m not going to say that XSD is the be all and end all of data modeling, because it isn’t. A lot of things are missing, which is why people had to invent things like Schematron, and why Microsoft is working on EDM and SML and so on. Model driven development is now on the peak of the hype curve so I expect that modeling will be a battle ground for a long time to come. So take a pragmatic approach to XSD - use it if it fits your purpose. Some folks use other modeling approaches then have a tool that spits out the XSD – and that’s fine too.

Same goes for XQuery and XSLT – I think of these as being yet another layer above XSD. We did XSLT and XQuery because we figured that XML is data therefore people will want ways to query and transform that data. Makes sense, and I’ve done a lot of XSLT development, I still use it for specific tasks, but some things are a bit tedious. I find myself escaping to script a lot. XSLT 2.0 is a good improvement, but again, these things are not going to be the be all and end all of query and transformation languages.

For example, I’m a huge fan of the work they are doing in VB 9 with XML literals connected to our new Language Integrated Query (LINQ). It makes a lot of sense, because instead of having to “escape” to script, you just write the code you need right there in place – you have a complete general purpose programming language at your finger tips. VB-XML integration is big leap forward for VB programmers and allows those developers, who may not be as familiar with the standard XML technologies like XSLT, to easily process XML data inside their programs. It’s a huge advantage to the VB programmer and we think it helps make VB an extremely compelling language for XML programming – it makes me want to write VB again, and I’ve heard many others say the same thing. However, it is VB-specific so development teams that need cross-platform interoperability at the query/transform layer are likely to stick to the standard technologies like XSLT and XQuery.

There’s a very interesting tension going on here where general purpose languages like VB and C# are moving up into declarative space with LINQ but not going all the way into declarative, versus SQL, XQuery and XSLT which are fully declarative with no side effects, which are therefore more optimizable, but sometimes rather incomplete as programming languages go and rather hard to author in some cases. I really don’t know how it’s going to end up. I think we should continue innovating on both approaches and see what happens. It should be very interesting.

As for all the myriad other XML standards out there, there’s a lot of hype that you have to sort though. To me it’s a funny thing to see programmers going to town making XML more complicated with layer upon layer of new concepts. I remember going to a conference and people were telling me “stop! – we can’t take any more”. There is genius in simplicity. I’m glad to see the renewed focus on simple REST-ful XML based services for this reason. If simple works, why complicate it. Conversely, if it isn’t simple, chances are people just won’t use it.

What can we expect from Microsoft in the future in the XML arena? Will support for XQuery 1.0 and XSLT 2.0 become part of Microsoft’s offering?
Microsoft is a pretty big place, so it’s hard for me to know all that is going with XML across the company. But I do know about .NET, Visual Studio and SQL server. As I’ve mentioned before we are shipping the XML support in Visual Basic 9.0 with XML literals, XML axis properties and integration with LINQ to XML. LINQ to XML is our API which we are adding to the XML runtime in .NET 3.5, it is a new XML object model that is designed to work well with the Language Integrate Query capabilities of C# and VB. We are also shipping some cool new features in the XML tools in Visual Studio 2008, including an incremental parser with extensibility API based on LINQ to XML that 3rd party XML designers can build on. We are also adding data breakpoints in the XSLT debugger and we have a new command line tool named “xsltc.exe” which takes XSLT and generates a .NET assembly which you can then deploy with your app instead of the XSLT source so you don’t have to compile XSLT on the server. Anton Lapounov has a great blog that talks about that. There is not much else new in the System.Xml runtime because Visual Studio 2008 is essentially a service pack release of the .NET 2.0 runtime, so we’ve fixed some bugs there. We are also working on some XML features in Silverlight and we put up a preview of our LINQ to XSD work on MSDN. We are working on a new XSD designer and you will see a CTP on MSDN pretty soon.

As for XQuery, you probably know we have a subset of XQuery already supported inside SQL Server. We currently have no official plans that we can announce on a client side XQuery engine but we are definitely interested in expanding client side query processing. LINQ offers a path to this (for both relational as well as XML). ESQL provides another client-side investment. We are open to customer feedback on the relative importance of client side XQuery compared to all these other possibilities. Meanwhile we are doing some XQuery improvements in SQL Server 2008, adding LET, better datetime support, and lax validation.

As for XSLT 2.0 - we’ve heard from customers and understand the improvements in XSLT 2.0 over XSLT 1.0, but right now we’re in the middle of a big strategic investment in LINQ and EDM for the future of the data programming platform which we think will create major improvements in programming against all types of data. But we are always re-evaluating our technology investments so if your readers want to ramp up their volume on XSLT 2.0 please ask them to drop us a line with their comments.

Meanwhile I was rather surprised by the positive feedback to my little XML Notepad 2007 tool. It now has over 1 million downloads and is still going strong. Not bad for a couple weeks work and no marketing. So something about this tool hit the sweet spot. The interesting thing there is it reaches out to the non-programmer community and I think that is the key and it has the right balance of simplicity and usability. A Swedish customer said it is “logam” – just enough. I think you should expect to see more from Microsoft in the future that helps to make XML something that everyone on the planet can deal with easily and in a way that integrates deeply with everything else Microsoft provides.

As the XML hype is wearing off, folks are realizing that not everything that made it through the standards process needs to be implemented. So I think you will see Microsoft continue to innovate on new XML technologies and tools like LINQ to XML and VB 9.0 XML and you’ll see Microsoft taking a more pragmatic customer-demand-driven approach to standards. Microsoft will probably never implement every standard that comes out but I’m confident you will see Microsoft continue to be committed to the really important XML standards, like XML 1.0, and any other standard that is essential to achieving cross-platform interoperability, including Open XML. There is enormous power in the cross-platform reach of XML and the huge industry adoption it has and I’m happy to see that Microsoft is continuing to do some really innovative work with XML.

Op 17 en 18 september is in ons land weer de Software Developer Conference (SDC) georganiseerd door het Software Developer Network. Hier zijn een aantal toonaangevende internationale sprekers te zien waarvoor je normaal gezien naar het buitenland moet. Zeker de moeite waard dus. Zie http://www.sdc.nl voor meer info. Ik weet zelf helaas nog niet of ik erbij kan zijn, hoewel Ordina wel een stand heeft.

Het Office platform is zo onderhand behoorlijk groot aan het worden en je weg vinden wordt dan ook steeds lastiger. Met de Microsoft Office Interactive Developer Map kun je de weg weer vinden. De Developer Map is een WPF applicatie met een overzicht van alle technologieen de van toepassing zijn op het Office platform. Door te klikken op een bepaalde technologie krijg je allerlei detailinformatie en documentatie, of wordt je naar het betreffende MSDN Developer Center gestuurd. De definitie van de "kaart" werkt zichzelf bij van de servers van Microsoft als er zaken veranderen dankzij het gebruik van WPF/XAML. Cool!

Een goed overzicht met tips van Scott Hanselman.

I've been working with ASP.NET 2.0 for a long time now and I thought I'd seen most of it. Today I found something that must have flown under my radar all this time: the Health Monitoring API. Functionally this API is similar to log4net and the Enterprise Library logging Application Block. Although both of these work well I prefer functionality that comes "out-of-the-box", because you don't have to do anything to get it, it works, and your pretty sure it'll be kept up to date by the folks in the product team(s). So even if you're using another logging mechanism, be sure to check out the Health Monitoring API. How To: Use Health Monitoring in ASP.NET 2.0 from the  Pattern & Practices group is a good starting piont.

Ik werk al zo lang met ASP.NET 2.0 dat ik dacht alles wel een keer gezien te hebben, maar nu blijkt er toch iets aan mijn aandacht ontsnapt te zijn: de Health Monitoring API. Deze API is qua functionaliteit vergelijkbaar met log4net en het logging Application Block van Enterprise Library. Hoewel dat beide goede logging biedt ben ik toch altijd wel een voorstander van functionaliteit die "out-of-the-box" beschikbaar is, omdat je daarmee zekerder bent van de levensduur en bugfixes. Ook als je een ander loggingmechanisme gebruikt is het dus zeker de moeite waard om even naar de Health Monitoring API te kijken. Een goed startpunt is How To: Use Health Monitoring in ASP.NET 2.0 van de Pattern & Practices groep.

Onlangs kreeg ik de volgende melding in Visual Studio 2005: The project file ' ' has been renamed or is no longer in the solution. Hierdoor was het niet meer mogelijk om ook maar iets van het project te bouwen. Het vervelende aan dit probleem is dat je geen idee hebt wat er nu eigenlijk aan de hand is. Na enige tijd vond ik uit dat dit gebeurt als een Web Project referenties bevat naar assemblies of projecten die het niet kan vinden. Je lost dit als volgt op:

1. Rechts-klik op het Web project en kies Property Pages.
2. Er verschijnt een venster met alle referenties, zowel naar de bin-map, GAC of andere projecten in de solution.
3. Verwijderer de referenties gemarkeerd met (unavailable).
4. Waarschijnlijk kun je nu niet bouwen omdat je assemblies mist. Als je de referenties daar naartoe maakt zou alles moeten werken.

We were recently faced with the error message The project file ' ' has been renamed or is no longer in the solution in Visual Studio 2005. The problem is that from this message you have no idea what is actually the matter. We finally figured out that this happens when a Web Project contains references to assemblies or projects it can't find. Here's how you solve this:

1. Right click the Web project and select Property Pages.
2. A window will open which lists all the references, either to the bin-folder, GAC or other projects in the solution.
3. Remove those that show (unavailable) behind it.
4. Chances are that now you can't build because the reference is not there. Simply add the reference again and you should be OK.

XQuery 1.0 en XSLT 2.0 hebben heel lang op zich laten wachten. Al drie jaar geleden wilde ik een boek schrijven over XQuery en toen was volgens W3C de specificatie bijna klaar. Dat "bijna" was dus een ruim begrip... maar goed, ze hebben nu eindelijk W3C Recommendation status verworven. Moet ik de voorgestelde inhoudsopgave voor m'n te schrijven XML boek helaas wel aanpassen :(.

Via het blog van m'n collega's bij Ordina Development Center Microsoft kwam ik deze aardige IE add-in (IEeee) tegen. Geweldig voor testers omdat ze vanuit IE de bug kunnen rapporteren. Daarbij worden de volgende gegevens toegevoegd aan het item in Team System:

• Screenshot
• HTML Van de pagina en alle frames/iframes
• Stylesheets en scripts die door de pagina gebruikt worden
• Details over de machine en de browser van de gebruiker

Microsoft heeft een security patch uitgebracht voor Visual Studio. Het "gat" in VS2005 wordt als kritiek beschouwd, dus is het heeeel verstandig de patch te installeren. Zie voor meer informatie en download Microsoft Security Bulletin MS06-073.

Alex Thissen meldt op deze blogpost dat het probeert om zoveel mogelijk te programmeren met alleen z'n keyboard, omdat keyboard+muis vertragend werkt. De assumptie daarbij is uiteraard

1) Je kan alle shortcuts onthouden.
2) Je hebt de shortcuts vaak genoeg nodig om ze te onthouden.

In beide gevallen moet ik helaas zeggen "lukt me niet". Vandaag de dag spendeer ik minder uren aan programmeren en meer aan zaken die om het ontwikkelproces heen liggen. Voor hardcode programmeurs is dit wel handig lijkt me, hoewel ik wel een beetje flashbacks heb naar discussies over waarom VI beter is dan Emacs (of andersom).

Nu Internet Explorer 7 uitgerold wordt op de meeste computers ontstaat de situatie dat sites zowel met IE6 als IE7 goed moeten werken. Dit testen vanaf 1 machine is lastig omdat IE7 over IE6 heen gezet wordt. Maar geen nood, je kunt altijd nog gebruik maken van Virtual PC om in een virtuele machine met IE6 te testen (of IE7 natuurlijk als IE6 op je eigen machine staat). Microsoft biedt hiervoor een kant-en-klare Virtual PC. Meer informatie en de nodige download links vind je op het IEBlog. Bij The Vision Web werken we al tijden met Virtual PC voor het ontwikkelen en testen van applicaties, dus waren we zelf ook al op deze oplossing gekomen. Het is echter wel erg handig dat Microsoft een standaard Virtual PC aanbiedit hiervoor.

Serieus over beveiliging? Dan is de Threat Analysis & Modeling tool een goede toevoeging aan je arsenaal. Met deze tool kun je in kaart brengen welke (mogelijke) beveiligingsproblemen je applicatie heeft. Versie 2.1 verbetert een aantal zaken, waaronder export naar Work Items voor Visual Studio team System en voegt zaken als Auto Save toe. Download Threat Analysis & Modeling v2.1

Het jaarlijkse TechEd "feest" is weer begonnen. Gisteren al met een etentje met de MVP's waar ik weer eens heb kunnen praten met m'n mede-MVP's, waaronder Andre Obelink en Sander Gerz die boeken over VB en C# hebben geschreven in dezelfde serie waarvoor ik net een boek geschreven heb (De Basis). Leuk om ervaringen uit te wisselen.

Vanmorgen de keynote van Eric Rudder, die vooraf gegaan werd door Simon Brown. Simon had een leuk intermezzo met Arfa Karim, de jongste Microsoft Certified Professional (11 jaar, uit Pakistan). Arfa liet een zelf geprogrammeerde calculator zien en de code zag er netjes uit. De demo's tijdens Eric Rudder's keynote waren ook leuk om te zien. Als je de afgelopen twee jaar met je kop in het zand hebt gezet waren ze zelfs indrukwekkend.

Later meer over de sessies...

Het gebeurt me nog wel eens dat ik een SQL Server database aangeleverd krijg en dan wil ik graag een diagram hebben van de tabellen en relaties. Zo'n diagram kun je maken door er een toe te voegen in de Database Diagram map die te zien is in SQL Enterprise Manager (2000) of SQL Management Studio (2005). Vaak krijg je dan de melding hieronder:

Helaas kun je de Database Properties in dat geval niet opvragen (in elk geval niet in SQL Management Studio), dus kun je het niet aanpassen. Door de volgende query uit te voeren is het euvel ook opgelost

ALTER AUTHORIZATION ON Database::[Database Name] TO [sa]

Kan met de Augustus CTP van Microsoft Robotics Studio. Zo te zien zijn er ook al heel wat (verkrijgbare) robots die hiermee aan te sturen zijn... leuk speelgoed.

One of the great things about a DataSet, from a code generation perspective that is, is that it is defined in an XML Schema with some added features for TableAdapters and such. I wrote a very simple code generator that takes a DataSet definition and an XSLT stylesheet and generates a code file. I also made it possible to call an XSLT stylesheet for each table in the DataSet so I could create Data Transfer Objects and such from the table definition. I had some trouble to get it going though because of a namespace issue. The namespace that you expect to be the default namespace isn't in actual fact as I'll explain with the following fragment:

<xs:schema id="TrackingTracingDataSet"
           targetNamespace="http://tempuri.org/MyDataSet.xsd"
           xmlns:mstns="http://tempuri.org/My.xsd"
           xmlns="http://tempuri.org/MyDataSet.xsd"
           xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:msdata="urn:schemas-microsoft-com:xml-msdata"
           xmlns:msprop="urn:schemas-microsoft-com:xml-msprop"
           attributeFormDefault="qualified" elementFormDefault="qualified">
  <xs:annotation>
    <xs:appinfo source="urn:schemas-microsoft-com:xml-msdatasource">

From the above fragment you would think that the default namespace is http://tempuri.org/MyDataSet.xsd, because the default namespace is declare with the statement xmlns="http://tempuri.org/MyDataSet.xsd. The actual default namespace is however defined in the source attribute of xs:appinfo element, so the default namespace is urn:schemas-microsoft-com:xml-msdatasource.

Updated August 13, 2006 to reflect Sander's comment.

On my old blog (in Dutch) I commented that I thought the WindowsImpersonationContext was clumsy and I wrote a replacement that you can use as follows:

Console.WriteLine("Current user: "+ WindowsIdentity.GetCurrent().Name);
WrapperImpersonationContext context =new WrapperImpersonationContext(domain, username, password);
context.Enter();
// Execute code under other uses context
Console.WriteLine("Current user: "+ WindowsIdentity.GetCurrent().Name);
context.Leave();
Console.WriteLine("Current user: "+ WindowsIdentity.GetCurrent().Name);

Recently a visitor noted that the code wasn't quite right (missng a view using statements, and I found some other small issues. The correct code (including namespace references) is below. It takes care of all that nasty calls into the Win32 API. The one thing you need to be aware of is that it requires permissions to call into a DLL (i.e. run unsafe code), which is why I added the attributes that indicate this. Unfortunately that renders this class useless in a hosted environment, unless you strong sign the assembly and pursuade the host to allow your assembly to run in Full or High trust. This however is a problem you will run into regardless your use of this class. As soon as you call LogonUser you need at least High trust. If this is something that should be possible under lower trust by default it's up to the folks in Redmond to add this functionality to .NET and handle it as such.

using System;
using System.Runtime.InteropServices;
using System.Security.Principal;
using System.Security.Permissions;
using System.ComponentModel;

publicclass WrapperImpersonationContext
{
   [DllImport("advapi32.dll", SetLastError =true)]
   publicstaticexternbool LogonUser(String lpszUsername, String lpszDomain,
   String lpszPassword, int dwLogonType, int dwLogonProvider, ref IntPtr phToken);

   [DllImport("kernel32.dll", CharSet = CharSet.Auto)]
   publicexternstaticbool CloseHandle(IntPtr handle);

   privateconstint LOGON32_PROVIDER_DEFAULT = 0;
   privateconstint LOGON32_LOGON_INTERACTIVE = 2;

   privatestring m_Domain;
   privatestring m_Password;
   privatestring m_Username;
   private IntPtr m_Token;

   private WindowsImpersonationContext m_Context =null;

   protected bool IsInContext
   {
      get { return m_Context !=null; }
   }

   public WrapperImpersonationContext(string domain, string username, string password)
   {
      m_Domain = domain;
      m_Username = username;
      m_Password = password;
   }

   [PermissionSetAttribute(SecurityAction.Demand, Name ="FullTrust")]
   publicvoid Enter()
   {
      if (this.IsInContext) return;
      m_Token =new IntPtr(0);
      try
      {
         m_Token = IntPtr.Zero;
         bool logonSuccessfull = LogonUser(
            m_Username,
            m_Domain,
            m_Password,
            LOGON32_LOGON_INTERACTIVE,
            LOGON32_PROVIDER_DEFAULT,
            ref m_Token);
         if (logonSuccessfull == false)
         {
            int error = Marshal.GetLastWin32Error();
            thrownew Win32Exception(error);
         }
         WindowsIdentity identity =new WindowsIdentity(m_Token);
         m_Context = identity.Impersonate();
      }
      catch (Exception exception)
      {
         // Catch exceptions here
      }
   }

   [PermissionSetAttribute(SecurityAction.Demand, Name ="FullTrust")]
   publicvoid Leave()
   {
       if (this.IsInContext == false) return;
       m_Context.Undo();

       if (m_Token != IntPtr.Zero) CloseHandle(m_Token);
       m_Context =null;
   }
}

This is a great tool that helps you create a sound threat model of your application, leading to a more secure application. Do yourself a favor and download it.

Eerder al berichtte ik over deze Threat Modeling tool in Eerste hulp bij Threat Modeling toen dit nog een release candidate was. Deze tool is er nu als final release en is hier te downloaden.

Ted Neward legt in zijn blogpost The Vietnam of Computer Science uit welke problemen zich voordoen bij object-relational mapping (de overgang van de database naar objecten en weer terug). Het stuk is behoorlijk lang, dus neem er even de tijd voor.

Threat Modeling van een applicatie, het identificeren van mogelijke bedreigingen en het registreren van de genomen maatregelen, wordt steeds belangrijker bij het maken van veilige applicaties. Zonder Threat Modeling is het eigenlijk niet mogelijk om te weten of een applicatie veilig is of niet. Aangezien Threat Modeling best lastig is, heeft Microsoft een tool om dit te doen, Microsoft Threat Analysis & Modeling. Hiervan is momenteel versie 2.0 Release Candidate 1 te downloaden, en binnenkort zal de uiteindelijke versie verschijnen. Deze tool bevat een wizard om je applicatie mee te definieren, waaruit de mogelijke threats worden gegenereerd. Zonodig kun je op basis van de Attack Library meer zaken toevoegen aan je model. Doe er je voordeel mee!

Overigens staat deze tool nog los van Visual Studio, hoewel je wel work items kunt maken en exporteren naar VS. Het schijnt de bedoeling te zijn dat deze tool t.z.t. onderdeel wordt van Visual Studio.

WinFS, het op de PDC 2003 aangekondigde relationele file systeem, is op sterven na dood. Oorspronkelijk zou deze technologie in Windows Vista komen en zou je makkelijk je bestanden kunnen organiseren. Op het WinFS blog van Microsoft is nu aangekondigd wat er gaat gebeuren met WinFS. In het kort: onderdelen van de technologie komen in de volgende versie van ADO.NET en SQL Server terecht, maar het zal niet meer geleverd worden als aparte technologie. Daarmee is de gedachte van het relationele file systeem min of meer van de baan zo lijkt het. De kritiek op dit bericht is niet van de lucht (zie comments bij het bericht), en terecht. Door deze koerswijziging is iets wat geweldig is voor zowel bedrijven als consumenten de nek opgedraaied en wordt het expliciet in de bedrijfssfeer neergezet.

See Scott Hanselman's blog for details (no use repeating his post): http://www.hanselman.com/blog/FreePowerShellIDE.aspx

Ik ben nooit zo van de command line geweest, maar Windows Powershell (voorheen bekend als "Monad") is wel heel cool. Om het makkelijker te maken om Powershell scfiprts te maken is het toch wel prettig dat er ook weer een IDE is. Zie deze post van Scott Hanselman.