Sitefinity 4.0 Beta running on Azure, now! - General Discussions - General Discussions - Progress Community
 General Discussions

Sitefinity 4.0 Beta running on Azure, now!

  • Sitefinity 4.0 Beta running on Azure, now!
  • Hi all

    Couldn't sleep last night so thought I would see how the current Beta got on with Azure. Turns out it seems to work fine already, have a look! :)

    http://sitefinity.cloudapp.net

    I just created a couple of pages. /home and /test and wrote a  tiny widget to get the instance of the Web Role that handled the request. So you can see which instance generated the page, there are currently 2 Web Roles running.

    As you can see the whole interface is there too :)

    http://sitefinity.cloudapp.net/Sitefinity

    I will leave it running for a couple of days while I do some testing and anything else I can think of at this early stage.

    It ran fine in the development fabric as well so all looks good. The admin interface doesn't seem to mind being load balanced without any configuration and is clearly not using any session state as you can get subsequent requests handled by different instances, cool!

    The DB also migrated to SQL Azure without any errors, well done Telerik.

    It went a little slowly to start with but I then realised I had put the DB in W Europe and the App in N Europe!! Once I located them together it goes very quickly I think. I am just running 2 small instances for the Web Role. Admin site runs quickly too.

    I haven't found anything that doesn't work yet, would appreciate the guidance of the Sitefinity team on what to try..? I would like to write a logging provider so the SF logs can be transferred to Azure storage along with the other diagnostics. Hopefully we can have an image manager provider to store images/videos/docs on Azure Storage which we can then push out to the CDN to speed things up further.

    What next, I want to develop this, I think SF on Azure is going to be great.

    Matt
  • First off, wow!  Great job.  This was on our "to do" lists but you beat us to the punch.  (Thanks for making us look bad.)  (Just kidding.)   :)

    --

    For anyone following in your footsteps, what did you need to do to get this to work?  Did you simply change the connectionString for the database?  Did you create the database locally and then migrate (by hand) this database to SQL Azure?  Or did you let Sitefinity 4.0 create the SQL Azure database (tables, etc.)? 

    Were any other customizations needed?

    --

    We've had some internal conversations about this.  One concern I have is the SQL Azure database limit (50GB).  This is a lot of storage for a single Sitefinity web site but, in theory, a single Sitefinity installation could host many different web sites.  With media being stored, by default, in the database this could present a scaling problem.  

    As you mentioned, Sitefinity is stateless, so I'm not terribly worried about the front-end servers.  I'm more worried about the data storage piece of this.  The 50GB database limit kills the promise of infinite scaling.  As you mentioned, long-term it would be nice to have SQL Storage providers for key modules (images, documents, videos).

    --

    Another random question:

    Does the Configuration editor work fine?  (This editor modifies XML files in ~/App_Data).  I'm also assuming this would only modify the XML files for that single server.  How would those configuration settings be synchronized between the other servers?

    --

    Anyway, these are my stream of consciousness questions.  Don't feel compelled to address all of this.  These are normally the questions I aim internally at the Sitefinity team.  But you volunteered...     :)

    Great job on this.  Very exciting.  

    Gabe Sumner
    Telerik | Sitefinity CMS
  • Hi Gabe

    Sorry, I couldn't wait to give it a go, have been waiting for 4.0 to play with Azure for ages :)

    I took a DB created in local SQL and migrated it using SQL Azure MW at http://sqlazuremw.codeplex.com/
    This goes through the DB and checks for any inconsistencies in the Azure subset of SQL server. I was very pleased that there weren't any, I tried this with a 3.7 DB a while ago and ran into lots of problems. It will also script the data and BCP it to the newly formed tables so might actually be a pretty cool deployment option for pre populated deployments.

    I tested it all worked by then pointing a standard local Sitefinity site to the Azure DB, bit slow but cool.

    I then created a new Cloud service project, copied all the Sitefinity code over from an existing project, fiddled with namespaces etc, merged the 2 web.configs to leave the stuff that needed to be there for a cloud app. You also need to leave files like WebRole.cs etc.

    I could then run it in the dev fabric and it worked fine, so just packaged it, changed the logging providers to Blob Storage and deployed it, all worked a dream.

    DB wise yes obviously there is currently the 50GB limit. This starts to get expensive too and while I'm sure they will increase the sizes it's not really the ethos of the platform. I would say we need to keep all heavy content like docs/images/video out of the the DB or at least in a different copy? You would get faster performance from 10 5 GB instances than one big one.... The ideal though is to have all this content stored in Blob Storage as this won't slow down as it grows huge (if we partition by user etc) and we can also possibly feed it into the CDN for even better perfomance.

    Yes the Configuration folder in App_Data is the main problem as it's read only, you get the following message when you try o save a config change:
    Access to the path 'E:\approot\App_Data\Sitefinity\Configuration\ToolboxesConfig.config' is denied.

    The only way round this is to provide a provider to store the config files on Blob Storage and serialize/deserialize them as needed as per here http://brendan.enrick.com/post/Storing-and-Retrieving-Serialized-Data-with-Azure-Blob-Storage.aspx This way any number of Roles can read/write them and they are not at the mercy of the instance recycling as well.. I believe projects like http://orchardproject.net/ are already storing app_data files that might change here as well. It would be much better place for the error log as well and is the best practice as all logging data from individual instances I copy to a Blob Store where you can analyze them at your will.

    I think you have the front end and DB sorted for Azure, think we just need to add a couple of providers to make the config run properly and allow storage in Blobs. You already had an Amazon provider videos for 3.7 as I remember so shouldn't bee too big a deal? Reading configs should be fine as it's just getting a stream from Blob storage rather than reading it from the filesystem?

    It would be a real shame if 4 wasn't completely Azure capable as it's so close, I think it will be an incredibly powerful product in the cloud.

    I'm happy to work to help make it happen, I have some grand ideas for it but needs to have the power of being able to run many front ends and have a HA SQL system behind it :)

    Matt









  • update:

    Just noticed the Analytics data seems to have vanished, still data on Google but none on Siefinity..

    M

    Update:
    So I'm guessing the Analytics module tries to store data locally? It seems to do odd things, I can't see any files that it's creating on a normal installation though.

    Started getting a spate of errors saying the DB connection was already closed. Remembered that using ORM with Azure you need to tell it to turn off MARS as it's not supported on Azure.  So I updated the connection string and redeployed it and it seems to have fixed it, although I don't know the effect of the re-deploy. Memory is a little fuzzy but think ORM has some settings to cope with checking to see if the connection is still open?

    Another really strange phenomenon was that after I redeployed the Web Roles there was no page set as the homepage......

    Will keep monitoring it for more issues.

    M
  • I'm getting an error when I launch http://sitefinity.cloudapp.net
    Did you unload?
  • Hi Krishna

    No it's still there, I think these errors are caused by a mismatch in what ORM is expecting and what Azure SQL is doing.
    Azure will actively close connections that are not being used and I think ORM is expecting them still to be open and trying to use them, hence getting this error.

    I remember something in the ORM documentation about a flag to get it to check connections are still open before it uses them.... I'm looking for it now :)

    Matt

    UPDATE:

    I believe it just needs this changing in the ORM backend config:

    <backendconfiguration id="azureConfiguration" backend="azure">
            <lockTimeout>5000</lockTimeout>
            <testOnAlloc>True</testOnAlloc>
            <mappingname>azureMapping</mappingname>
    </backendconfiguration>

    Its the <testOnAlloc> setting I think we need.

    In the ORM docs it says:

    Test connection before use - when this property is set to True, then each connection is validated before leaving the pool. This may have a serious negative impact on performance, so this property is set by default to False. The name of this setting in the App.config file is <testOnAlloc>.

    Going to mean more calls but think this is whats doing it.

    Now just need to find where in Sitefinity I can modify the ORM configurations...

    M
  • Hi all

    I've temporarily taken the app down while I get ORM singing sweetly with SQL Azure. It would work fine most of the time but then would cause a few errors with these closed connections.

    Stay tuned :)

    Matt
  • Have you any details on pricing that the installation and running costs will incurr in comparison to hosting on a provider such as DiscountAsp.Net.

    Regards,

    Neil
  • Hello,

    Matthew, your initiative is just great. My regards about this. The experience you share is certainly very helpful and we are all happy because of the results. 
    Please keep us posted, and cheers from the team.

    All the best,
    Georgi
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  • Hi nenwmn

    The answer to your question is lots more! I think $200 a month for 2 Web Roles (small instances), a 1GB DB and a Storage account for log files etc is probably ballpark but thats a theoretical calculation and pretty much the minimum.
    However, I don't think you can really compare a shared hosting account with the Azure platform, they are different ends of the hosting spectrum. 

    When you buy a shared hosting package you are typically buying a website on a shared server and maybe a SQL database on another shared machine. It obviously varies how much contention there is but I have used shared providers (albeit not for a while) where there were 200 SQL DBs on a single server! At the end of the day if you are paying $10 a month you can't complain.
    As well as just sharing 1/200th of the resources of that server you are at the mercy of other events like the server being rebooted because someone else has caused a problem. In the event the server fails, the hosting company is going to have to restore all the data from backup, at which point you are just one of 200 people asking when your data will be available again.

    With azure you are basically paying for your own dedicated servers on demand, so you are not going to get the webserver going very slow or the DB server being rebooted because of someone else. However the Azure services go much further in terms of availability, resilience and flexibility.

    One of the main attractions of Azure is the fact that it is so highly available. Take SQL Azure as a good example, it has a 99.9% SLA because of the way it is designed. The Azure system stores 2 copies of your database (in different places) and applies any changes you make to both copies in real time, so it has 2 current systems at any one time. Say the currently in use one fails, then the system will fail over to the other copy and carry on. You will be unaware of this, as will your apps. To get this, even in a dedicated hosting environment you are looking at clustering/transaction log shipping (so multiple servers) or other virtualisation plan such as VMware with VMotion so that your server is not dependent on one set of hardware (if the server fails the machine will migrate to new hardware without stopping).  When you start to look at the costs of these plans, then Azure looks very good value because you don't have to invest up front in the infrastructure but also you don't need that in house expertise of clusters, SANs, load balancers etc.

    Finally with cloud systems like Azure you have the elegance of ultimate scalability, not just for day to day loads but for development, testing etc.. Day to day you can double your webserver count instantly from MMC or the web portal, no need to worry about building servers, deploying code, adding to load balancers! Scale them up when you have busy times, down when things are quiet. There is a great example here of how you can create apps that you just couldn't do on traditional dedicated server systems. (They were starting up hundreds of SQL instances and Web Roles for a ticketing site when selling tickets for large stadiums which causes huge spikes in traffic for a few minutes!)
    Development wise it gets even better. If you want to test out how a new version of your code is going to work under exact live conditions, then you can deploy a clone of your site and test it under load so you aren't going to get any surprises when you go live.

    So in summary, I think Azure is aimed at the dedicated server market rather than the shared hosting market. The costs are higher but the benefits are so much greater as well.

    Regards

    Matt



  • Hi mattc,

    Really appretiate the response given and some excellent pointers to those not familiar with Azure, your experience has already helped me.

    Best Regards,

    Neil
  • I also feel that Azure would be a very valuable 'first level' supported deployment target for SiteFinity 4.  To that end, I will also offer my development time to help make that happen.

    I use Telerik products extensively, especially the ORM and WPF libraries, and I think that we could suggest targeted changes to the distribution to support cloud deployment.

    For example, I have often thought that reliance on App_Data was a bad idea for configuration settings.  It seems that there should be a true/false toggle in the web.config to store these types of settings in the database (the default) or in App_Data.

    In addition, when it comes to the database, I would prefer that you support the option to store all image media in a separate database anyway.  Even for local installs, I believe it is better to store that larger BLOB stuff in a separate database.  Ideally, I would also hope to have an option to store image media in a file system, with a reference to the path in the database (using smart 3rd normal form structure for directory and file so that a given media element only has a single integer foreign key i.e. File_ID).  I've coded this kind of structure before several times, and it is not that hard.  The nice thing is that it makes it easier for customers who use shared hosting where file system space is generous (and cheap) -- which is the norm -- to take advantage of that.

    To drive this point home, I have 2 TB of image and video files that I'd like to index in SiteFinity 4.  That covers more than 500 directories and about 100,000 files.  I'd like to retain the file structure in the database, because the hierarchical paths mean something about what the media is and from when.  If I wrote an import script to call the SiteFinity API, I would also use that structure to affect things like tags and categories.

    Anyway, I will strive to duplicate Matt's Azure installation and contribute to the process.
  • Hi Brian

    Glad to hear that there are others outside of Telerik who are very excited by this! :)

    Having contacted the Sitefinity team re a couple of things I found as a result of this deployment I know that they are soon to start getting to grips with the Azure support for 4.0. I was really impressed that it seems almost ready and yet it hasn't been suggested as being supported in the Beta. So think there will be lots of exciting work going on over the next weeks building up to a RC release.

    Personally I think that allowing the  config and assets to be stored in the DB and/or on Azure or other Cloud provider blob storage is the way to go. As you say with a switch or two it could be deployed in either mode. Lots of these changes would better suit a web farm environment as well as the Cloud one.

    I think the providers for storing content outside of the main DB will be along shortly, if not in the release then soon afterwards, or if not the community can write them once we get hold of the SDK. @Rok_b tweeted yesterday that he had already done some work on an Azure Blob storage provider for RadEditor http://is.gd/ebsgf

    Sitefinity is being developed with extensibility as the mantra so I'm sure just about anything will be possible.

    Regards

    Matt



  • Here's one more big fan of cloud services.

    And from a business perspective, I can only give two strong recommendations
    - provide a "ready to fly" Sitefinity environment on WindowsAzure
    - give new users/clients a chance to
        - easily create a "website"
        - link a domain to it (comparably like BPOS handles that with the hosted Exchange services)
        - provide a consumption pricing with different service levels.


    Third :)
    If you are not looking at this as an option, let's take Matt and our team and Microsoft to the next round table and get this started as a partner offering. that you can back through a dynamic pricing (1 ct per served page ?)


    Kind regards,
    Markus