High-Memory Quadruple Extra Large Cache Node (cache.m2.4xlarge): 68 GB memory, 26 ECUs (8 virtual cores with 3.25 ECUs each), 64-bit platform, High I/O Capacity cost 2.24$ per hour or about 1600$ per month!

For about 5000-6000$ you can purchase server with following hardware:
4x AMD Opteron 6128 / 2,00GHz Eight-Core (i.e. 32 CORES)
64GB ECC registered DDR3
2 * 80GB 2,5 Intel SSD with Adaptec RAID
1+1 1400W redundant (Hot-Swap) PSU
etc.

Now install here your favorite free Linux and Memcached and you are ready to go

Totally:

Server with better hardware (32 real 2GHz cores instead of 26 ECUs i.e. 8 virtual cores in AMZ, very fast DDR3 memory, redundant power supply and brand MB (Supermicro) instead of “well know instance lifetime issues in AMZ” etc) will cost you something like 3-4 month of AMZ instance rent! Sure, you can say like: what about Collocation cost? Hm… 1U server probably MAX 100$ per month (including some big traffic) etc.

Instead of Conclusion:

So you still think same about Amazon ElasticCache ???

Most probably you already store your virtual disks not in the system drive (i.e. not in drive ‘C’ for example), which is right approach (for performance and other considerations)

However not everybody know that in the C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines folder, Hyper-V actually store huge files (*.bin) for each virtual machine that you run on your Hyper-V host! That may be huge issue in case, if you have a lot of virtual machines in use and your system drive space limited. In addition, it also may introduce fragmentation problems, due to the fact that such files maybe created and removed each time you run your instance or when you create some snapshots etc.

So basically there are 2 main questions you may ask:
a) Why such files (*.bin) are created? And why they are so big? You may notice that size of such files actually equal to amount of virtual memory which you allocate for the virtual instance. So answer is simple – it just a place (such file) to store memory dump for Virtual Machine (instance) and required by Hyper-V
b) Can I get rid of such files? Answer – no In current version of Hyper-V it is not possible. However it is completely possible to move such huge files to separate disk(s) out from system disk! That SHOULD be done I think in any Hyper-V host anyway, in case if you have additional disks here. It is very easy to do that: on the first step of Wizard to create Virtual Machine, do not forget to change location for virtual machine from default one to location, where you have enough space to store both virtual machine memory dump and possible snapshots (in case if you plan to use them sure thing).

a) Rough ETA x 4

OR

b) Periodical updates of ETA

I found both make sense to use together, so I recommend something like this:

1. Initially Estimate Rough ETA x2 or x 4 (depends how you feel yourself confident with given issue), with precision to hour / day (round value up).
   Forget minutes precision in all cases, forget hour precision in many cases. If you feel that issue takes 2 minutes to fix – put estimation 0. If you feel that it’s more like 15 minutes – put estimation 1 hour. If you feel it’s like 3-4 hours, put estimation 1 day. Don’t try to be precise. Instead it should be realistic.
   For example you think it takes 20 minutes to fix issue but you don’t have previous experience with such issues, so you got 20×4 = 80 minutes and round it up to 2 hours. Put value 2 hours into issues tracking software.  If your software support ranges, put here 20 – 80 minutes range.
2. If you use estimation precision as hour, you should update your estimation at least every hour, if you see that now you can measure ETA better.  If you use precision day, update your estimation at least ones per day. As more frequently you update your estimation as better. Sure don’t be too paranoiac - update previous estimation only if it changes significantly.
   For example say you estimate issue initially to 2 hours, so after first hour you working on issue, compare your initial estimation (2 hours) with what you think you have now (don’t forget about ETA x 4), for example now you feel that you need another 35 minutes to fix, so you multiple 35 * 4 and got 140 minutes = 3 hours. So you adjust your initial estimation from 2 hours to 4 hours (1 hour you already spend plus 3 hours you think it will take more).

Advantages:

1. You update your estimations frequently so even if you made serious mistake in estimation before, you quickly resolve. Fail quickly Your colleges and boss know where you are, so excellent transparency and you agile
2. You don’t care if you need to go to made coffee, go to rest for a while etc. You made your estimation SAFE enough (x2 or x4!) and you know that in next hour (or day if bug is complex) you always may update estimation again with more precise value.

In contrast to approach above, it can better in some cases (in most of cases in my experience)  to use “relative” measurement instead of time measurement.  Unfortunately in reality, seems it’s a bit more hard to begin with in some less agile teams. Another issue is that it’s just not every issues / bug tracking software support that.  In that alternative approach you think how much more complicated given issue is, compared to other issues you have / you done before and made your estimations relative.

P.S. Many people don’t like x4. Sure it’s completely optional. I saw people (and I am one of them ) that can do exact estimations in most of cases: in some cases because of luck, in other cases because of experience. In different cases because of both. Maybe it’s because usually they (and me) do that calculation x2-x4 in the mind and do not do it formally.
Taking to attention fact that it’s impossible to work 9-12 hours per day with same focus (hope my boss don’t read that Joke), it’s best way to always remember that effectively you have about 4-5-6 hours “crazy” time per day when you really can do your job with high focus and with maximum performance, while other time is required to prepare yourself for that “fight / micro-sprint”. So at least x2 should be applied in most of cases to be safe and I recommend to do it in your mind so your colleges get only one final estimation value, not intermediate result. It’s not about transparency. It’s more about ‘useless information’ for everyone, how exactly you calculate your estimations. Everybody may value you only by how your real time to fix issue was different to what you estimate.  And in most cases if that is not important! But what is true is that nobody care how you calculate ETA after you made your job done. People trust you because of results of your work, not because you do nice calculations, but it takes you 10x times more to fix issue than you think initially.

Just keep it simple, as always

Projects like Simple Cloud API (http://simplecloud.org) in PHP, together with Zend Framework and many other PHP related stuff  sometimes KILLS me, .NET developer / architect!

Why, WHY we do not have something like Simple Cloud API in .NET??? Or at least it’s not well known! Is it so hard to build and promote it ?? NO! Why it takes whole a lot of time to introduce ASP.NET MVC, if we do it so many years in Java or PHP !? Great .NET framework, tons of projects, but so limited amount of “innovative” open source projects in .NET!

Take a look, most of famous .NET open source projects go in same versions or even few versions behind if compare to corresponding projects from Java (see Lucene.NET for example), but do not introduce really “innovations” in the software programming! Some projects do not even exists in .NET space like Solr (i.e. you should use Java VM to run it)! Some projects like ASP.NET MVC comes much later other MVC frameworks in Ruby (RoR), Python (Django) or even PHP (Zend Framework)… And Microsoft last time spend whole a lot of effort to bring Java  / PHP into Azure… instead of increase sponsorship for example innovative OSS projects in .NET and by this push developers to go for .NET from other frameworks!

Should somebody build library like SimpleCloud in .NET ASAP!? YES! (please don’t tell me that I should build it ) Why?

Too many companies really need same thing to be able to abstract away Windows Azure or any other Cloud specific solution using some open source  libraries (like SimpleCloud in PHP) and made solution portable across Clouds, before they spend $$$ and stuck with one Cloud supplier forever See my post about this!
And too many developers really expect .NET platform to be more innovative !!!

P.S. sometimes innovations can waste your time / effort, if parent company stop support it – see my post about Silverlight

Why this happens!?? Here is my thoughts:

1. Silverlight, like many other interesting concepts from MSFT was implemented with many technology related issues in early versions!
   Performance and fonts issues I can name as most important! I do NOT like how Silverlight apps looks at my TFT monitors (because of fonts issues), same like WPF applications! Sometimes, when I scroll something, it push me think that I should go to doctor to check my eyes again! (and same issues with fonts I see with IE9 beta, not sure MSFT will fix it someway or not before final release)
2. Silverlight supported on much less devices than HTML / CSS / JS / Flash and it seems like because it’s proprietary this can’t be changed in near future!
3. Development of Silverlight can be done right only using Microsoft tools like Visual Studio and Microsoft Expression Studio (some apps from it) that cost $$$
4. If you know Java, Python Django, RoR, PHP, Grails or any other language / platform / framework for web development, it’s much more easy to use HTML5 because you already know HTML / CSS / JS stack (as well as many web frameworks) than to move to something new like XAML / C# etc! I.e. basically only developers in .NET can consider using Silverlight, not other wide audience of Web developers! And while I can’t sure thing compare power of Silverlight and HTML5, MOST of required things can be done easy with nice JavaScript frameworks (think jQuery), HTML5, CSS3 etc.
5. Silverlight apps are always have big size! Sometimes there are even so heavy that web users probably will not wait so much time! Compare it to current compressed Javascript code and HTML5 compressed using standard gzip compression and / or different other compressors! What to feel it? Go to http://www.microsoft.com/windowsphone/en-us/buy/7/phones.aspx and check how long it takes to really get it in your browser!

I can give many other arguments around it, but generally when Silverlight out I made my choice – AVOID using it (as well as WPF, but that’s other story) for any live projects, until I will really feel the need of “all in one” web client application with really rich and heavy graphics / Video / Voice requirements!
It’s real fact that MOST of flash based sites, where site COMPLETELY was made in Flash FAIL, unless they touch some very specific area! I.e. it so easy to see that web users want “light” pages (HTML / CSS / Javascript) and not heavy as Flash / Silverlight get produced!
But again: I DO see big future anyway for Silverlight as really best framework to build RIAs out there in .NET space! More so, now it seems like Silverlight can take it place in WP7 development!

Just always you should ask yourself – are you sure you want to build (or customers ask you to build) RIA? Maybe you want to build super light, not heavy Web 2.0 (3.0) web site? Or may be you want to build something using only open / cross-platform technologies?
Than don’t even think about Silverlight, at least for now!

It seems like if you working in some management position, and must predict technical future for you company (or your own) business, it’s very high probability that you already start thinking same way like companies / start ups I describe above or at least start doing huge research in that direction!

But as Mark Twain said “Whenever you find yourself on the side of the majority, it is time to pause and reflect.”, simply follow “everybody” movements can lead to significant issues in the future!

So let’s quickly discuss following questions:

(1) when and why it make sense to move into the cloud from technical / business point of view
(2) how to do migration right and “safe” way

There are a lot of well known publications about (1), because every Cloud supplier (Google, Amazon or Microsoft for example) describe that in details to gather potential customers from regular hosting / collocation providers or to push customers that own already some server farm to migrate into the Cloud. Usually answer will be something like “you should do it as soon as possible, because it increase ROI, increase scalability, decrease maintain costs … bla .. bla.. bla..” Sure in some way they are right, i.e. for most of small / middle size businesses it’s DO make sense to migrate into clouds, unless they already invest a LOT into existed infrastructure (both hardware and software). But what is not acceptable is fact that some people (management, but not without pushing from low levels sometimes, i.e. from engineers) start really think / believe that (2) really does not make any big difference! I.e. just fact that you start migration and understand why to do it ASAP (ROI, bla bla bla) will lead your business to new levels, will save you a lot of money, will scale you to Google scale etc!
WRONG! It make absolutely critical difference HOW you migrate (2)!

Why its SO important HOW to do it? Lets give few reasons:

a) Most of cloud solutions require payments BY amount of requests (see Azure, Amazon RDS, Google storages etc), not just hourly rates!
And this can KILL some kind of businesses, if not to say MOST of the businesses!
It simply keep telling you something like: “you can’t scale PER customer actions because for each action from now on you should PAY”!
You pay when your customer come to each page that require trip to the database! Its not, it’s ABSOLUTELY different to situation that you have with hosting / collocation! From now on, you should optimize you data access, message queues not just to be able to scale, but to save COSTS for requests! Yep, most of clouds give you very CHEAP storage space, but they get much more from you because of millions of requests you get from customers that do not pay you PER page view in most of the cases!
Think for example situation that you have jobs board and your customers (companies usually) pay you when they add new vacancy! Does it make sense for you to pay for millions of search (browsing) users (you pay per each request to Cloud supplier), if you get money only from few people who post vacancies? And you should pay not only traffic (like it was with hosting providers), from now on in the cloud you should pay for search engine queries (requests to Solr / Lucene on multiple nodes for example), for MapReduce engine, for requests into database storage  (distributed hashtable or SQL database engine.. you still pay PER request) etc!

So? Before even consider moving into cloud you should think about all above and check is it will work for you! Even if you absolutely sure that it will, you anyway should care A LOT about code quality and additional features like distributed caching, replication effectiveness, traffic compressions etc! You should in most cases avoid many of such “hard” things in regular applications, but you simply can’t ignore them for most of real life application that should work in the Cloud!

b) You should migrate (or develop from begging) systems the way that they will be able to work OUTSIDE cloud! Why? Easy!

You do not have FULL control over your cloud supplier:

- It possible that supplier (Amazon, Google, MSFT or any other) will change prices or add additional fees
- It’s possible that you will want to migrate to different supplier tomorrow because it’s cheaper, or buy your own servers farm or / and even want to run OWN cloud (see Cloud.com).
- It’s possible that you will figure out that you was wrong with (1) above, i.e. you was think that you CAN afford to pay PER request, but was wrong and your servers get so huge traffic that your bills to cloud supplier quickly become not acceptable!
- etc

In all situations like above, it’s CRITICAL to be able to made a SWITCH! And that is what most of companies MISS! They simple couple technology into one Cloud supplier and did not have “back door” to quit!

From technical point of view (but sure it’s theme for different, dedicated post) you should design your architecture the way that your solution will work at least on dedicated servers farm in addition to selected Cloud. If you start using Azure Table Storage, Amazon SimpleDb or another other “cloud” specific data storage, make sure that you can execute your code on for example MongoDB, RavenDB or Cassandra and only then add support for Cloud storage! In case if you already have system that work locally (i.e. on regular dedicated server), you should make sure that you do not “broke” this functionality when start migration! And so on…

Sure it’s not all, I have much more to say about it. But it’s just enough to start at least think right way about life in the Clouds

There is also another, more extended version like:
“There are only two hard problems in Computer Science: cache invalidation, naming things and off-by-one errors”
(get more about off-by-one errors if you do not know WTF is it at http://en.wikipedia.org/wiki/Off-by-one_error … See? 100% you done it many times )

Such problems are really hard or close to impossible to solve! Naming, caching invalidation, collisions in a hash tables (OK, you can’t solve it completely, but you should get know it anyway! See Pigeonhole principle http://en.wikipedia.org/wiki/Pigeonhole_principle), race conditions, etc…

Can you name few more that made you put your hand over the wall last time?

So now we have whole a lot of different app stores to publish our apps from all major players like Google (Android, Chrome coming etc), Microsoft (WP7 for example), Apple (iPhone / iPad etc), Mozilla (coming), etc…

Interesting enough that most of them are big enough to take our (developers) attention because of really wide audience of potential app users!
It simply does not make sense anymore to ignore ANY of them

And what is interesting that “App Store” (or better to say “digital distribution platforms” as “App Store” actually Apple trademark…) business model (i.e “software FROM web to your device or specific OS or specific browser”) actually looks orthogonal to SaaS (i.e. “software as a service” or “software IN the web”, i.e. software that works in almost ANY browser powered device) model! Or not!??

Should we go in both directions together? Yep
But… How hard to grow in both!!!???

Let’s try to found some analogies in other areas to WWW…  One that come quickly to my mind because of same “network” characteristic is “biological neural network”…

Billions of neurons (web sites pages or rest / soap services in WWW with some very small peace of data if we compare to all data in WWW) connected to each other (using hyper-links / endpoints), now with video, voice, metadata and another kinds of “meta” information inside each node…

Same like biologic organism, when try to search for some data in the memory use neurons connections, search providers that work in WWW,
like Google or Yahoo or Bing use connections between pages and metadata to found any sort of data in WWW
If some page not connected to another pages in the Web, Search Engines will not found it (unless owner of such page add link to search engines manually… but even doing so, search engines will show this link in last position in search results or do not show at all)

Semantic Web 3.0, will for sure increase power of such search capabilities…. But!

What will be really big change, say in Web N.0, is ability of some “systems” in the world to use WWW same way like our brain currently use network of neurons… I not speak about simple or not simple search algorithms used by companies above and others!  If we start thinking about WWW as some network that was already trained by billions of WWW users (by us, i.e. by humanity), that already store most, if not all, of humans knowledge, we will understand that some sort of IA  already there….

So what WWW already have:

a) Knowledge: data and metadata, much more than any man can get into!

b) Some learning capabilities – WWW “learn” using us (humanity) as sense organs! I.e we, people, all over the world submit information into www and “teach” it during decades! And it’s not far a way, when we will see more devices that collect and submit different data into WWW without people interactions!

c) Communication – WWW is all about it, you know

d) What you think about perception? Maybe yes, who know? We can’t prove it for now , but we can’t tell that it does not exists! And sometimes we all get a feeling that it’s already there!

e) Ability to move and manipulate objects – remember “terminator” movie and SkyNet ? It’s there already! For sure now it’s possible to manipulate things in real life from WWW and everyday we see more and more of such things!

etc…

So some (if not all) of IA problems was solved in the same moment when WWW was born or in past few years!

The only questions are: how to made first Big Bang !!!????? And do we need to made it !!!???

There exists also not so good news – this time for PHP developers – support of PHP was removed from release of free (“community”) version of IDEA, so developers can stick with other completely free IDEs this time – for example NetBeans comes to my mind this time… as alternative… but after Oracle buy Sun, I not sure it is right choice… but this is “idea” for another post…)

Actually personally I don’t understand why “Community” version does not support PHP, Javascript, Python or Ruby as this languages are used by a lot (if not most) of “open source” developers that for some reasons does not feet into JetBrains licensing of commercial version of IntelliJ IDEA for open source projects. I think “vice versa” if compare to JetBrains – Commercial version must have extended support for Java development, while Community edition must support MORE other open source frameworks / languages, like Ruby, Python or PHP etc. Sure it’s only my “personal” opinion, but I think most of developers will agree with me – people that use IDEA for Java development (usually in enterprises) can (and will!) simply BUY commercial licenses, while a lot of potential IntelliJ users actually want to use it for small open source projects in non-Java languages stack and want to get IDE for FREE!

But totally – all new features show that JetBrains go in right direction – support for most “progressive” and “latest” technologies in IDEA (even if it is available only commercially )

Keep it going, JetBrains!