Speed ( Network )

Data centers are becoming heavily virtualized and increasing demand of video content is making 10GbE not  fast enough for toda’s Data Centers. You can expect 10GbE connections between servers within the same rack and 40GbE from rack switch to the core network switch or router or edge router.

SDN ( Software Defined Networks ) 

Your IT guys can quickly provision new servers and new storage but what about network as that’s not easy today, because each server they create they need to  manage network address, firewall settings and some other network parameters based on the organization network and security policies. Its tough job to automate network scalability, reliability and security. Openflow enables you to control the behavior of your network devices. Additionally openflow creates a common networking platform and you don’t have a lock-in.

Sustainability ( Green )

Not surprising at all that the power consumption used in data centers is more costly than the cost of the hardware  during its life time. Additionally organizations are facing increasingly pressure due to environmental implications of running these data centers, and regulatory requirements are increasing every year. You will see more or solar and bio-fuel, heat from servers can be used to power chillers that cool the water to be used in data centers. Using tools to monitor all data center resources will get more popular.

The kind of issues we today in Data Centers will drive most of the innovations in the future. Here is a current survey done on Data Center issues.

Some of the best information that can captured graphically is here:-

What is Amazon Glacier?

New, long-term, low-cost, secure and durable storage for data archival and backup.

How it integrate with AWS ecosystem?

What does it offer? What’s new?

Data such like digital media archives, financial and healthcare records, raw genomic sequence data and long-term database backups as being ideal for Glacier

How much does it cost?

Very low cost as compared to any other cloud service.

$0.01 per gigabyte per month

No upfront commitment

Storage Price : $0.01 per GB / month

Request Price :

UPLOAD and RETRIEVAL Requests $0.050 per 1,000 requests

LISTVAULTS, GETJOBOUTPUT, DELETE† and all other Requests Free

Data Transfer IN               Data Transfer Price:

All data transfer in               $0.000 per GB

Data Transfer OUT

First 1 GB / month               $0.000 per GB

Up to 10 TB / month           $0.120 per GB

Next 40 TB / month           $0.090 per GB

Next 100 TB / month          $0.070 per GB

Next 350 TB / month          $0.050 per GB

What will change?

Glacier is definitely not about short-term planning, this is a brand new approach to data archiving that is designed to help maintain information throughout the decade and century. Glacier seems to be perfect fit for enterprises that have long-term goals in mind.

In a long term, essentially you can see this as a replacement for tape

Conclusion

This solution targets large organizations with regulatory compliance, and media,medicine and entertainment companies with large amounts of digital storage.

Amazon said that in “coming months” it will allow customers to “seamlessly” move data between S3 and Glacier based on data lifecycle policies

Challenges

Migration of data

Bandwidth related cost with your current hosting/data storage environment

But again, keep in mind that there is nothing else that is more reasonably affordable solution than AWS Glacier.

Need to know more …

Amazon Glacier

Amazon Glacier: Archival Storage for One Penny Per GB Per Month

AWS Glacier’s dazzling price benefits melt next to the cost of tape …

Instance configuration that Amazon has just added:

High I/O Quadruple Extra Large Instance

60.5 GB of memory
35 EC2 Compute Units (16 virtual cores*)
2 SSD-based volumes each with 1024 GB of instance storage
64-bit platform
I/O Performance: Very High (10 Gigabit Ethernet)
Storage I/O Performance: Very High**
API name: hi1.4xlarge

*8 cores + 8 hyperthreads for 16 virtual cores

 

**Using Linux paravirtual (PV) AMIs, High I/O Quadruple Extra Large instances can deliver more than 120,000 4 KB random read IOPS

and between 10,000 and 85,000 4 KB random write IOPS (depending on active logical block addressing span) to applications.

For hardware virtual machines (HVM) and Windows AMIs, performance is approximately 90,000 4 KB random read IOPS and between 9,000

and 75,000 4 KB random write IOPS. The maximum sequential throughput on all AMI types (Linux PV, Linux HVM, and Windows)

per second is approximately 2 GB read and 1.1 GB write.

For those who want “Hard Disk Drive Vs SSD”

Like a hard drive, an SSD is a device for storing information. But unlike a hard drive, it doesn’t have any moving parts.

Today’s SSD are built with flash memory — the same stuff that stores data and applications on your iPhone.

These drives have been around for years, but they’ve been slow to make headway in the real world, in part because

they’re more expensive than traditional hard drives. A 300GB flash drive sells for around $500, whereas a comparable

hard drive is closer to $100. A 3 terabyte hard drive — which is about ten times larger — sells for around $350.

                                                                                                                                                                    Source: Wired

Microsoft’s Trustworthy Computing Division conducted research among small and midsize businesses (SMBs), defined as companies with between 100 and 250 PCs in the U.S., India, Hong Kong, Malaysia and Singapore. The goal of the study was to gather data to better understand the cloud’s benefits relating to IT costs, improved security and better time management. For the purpose of this research, cloud users were defined as companies that used a cloud service via a subscription model.

“The security benefits of the cloud are there to see,” said Adrienne Hall, general manager of Trustworthy Computing at Microsoft. “In addition to driving towards industry standards such as the Cloud Security Alliance’s (CSA) Security Trust and Assurance Registry (STAR), we must step up our efforts to help SMBs, and other much larger organizations, better understand the security benefits of the cloud so they can make informed decisions about the suitability of the cloud for their business.”

The study illustrated one such mobile content delivery provider, SkyWire Media, reports positive results after cloud migration. The company introduced Microsoft’s cloud management service Windows Intune that help them into cost and time savings and better protection against cyber threats.

There’s a perception that security is a barrier to cloud adoption,” said Adrienne Hall, general manager, Microsoft Trustworthy Computing, “Yet when companies embrace and invest in cloud services, they find the benefits far outweigh previous concerns.”

]]> http://www.blog.yogeshmalik.com/smb-summary-of-india-cloud-study-results-may-2012-microsofts-trustworthy-computing/feed/ 0 http://www.blog.yogeshmalik.com/the-nexus-of-forces-social-mobile-cloud-and-information-gartner-june-2012/?utm_source=rss&utm_medium=rss&utm_campaign=the-nexus-of-forces-social-mobile-cloud-and-information-gartner-june-2012 http://www.blog.yogeshmalik.com/the-nexus-of-forces-social-mobile-cloud-and-information-gartner-june-2012/#comments Wed, 27 Jun 2012 07:50:36 +0000 blogyogesh http://www.blog.yogeshmalik.com/?p=1459 The management of this complexity falls on the ecosystem of technology providers — from enterprise IT shops to cloud service providers to device manufacturers — and the ease with which pieces fit together and flex around unforeseen scenarios. The task of enterprise leaders is to hide complexity under a layer of simplicity. 

Detailed Research  Source: Garner

“Orbitz found Mac users on average spend $20 to $30 more a night on hotels than their PC counterparts, a significant margin given the site’s average nightly hotel booking is around $100, chief scientist Wai Gen Yee said. Mac users are 40% more likely to book a four- or five-star hotel than PC users, Mr. Yee said, and when Mac and PC users book the same hotel, Mac users tend to stay in more expensive rooms.

“We had the intuition, and we were able to confirm it based on the data,” Orbitz Chief Technology Officer Roger Liew said.

The intention is to allow the creation of a single template for a stack and use it across different cloud providers or cloud instances. The template could be a simple server specification (OS, Memory, HD, etc.), a platform, or a complete application environment. Once the template has been defined, a brokerage engine will use policies to select the best cloud service to create instances of the stack.

For example, if a customer had created a standard developer stack (e.g., based upon the following components: Linux, Apache, MySQL), the brokerage capability would be able to instantiate this stack on any of the customer’s cloud service providers. Initially, there will be only one provider but in the future we would look to partner with a number of market leaders offering public and private cloud services

Actions based on policies

Examples of action-based policies might include:

• When the CPU load increases over 80% for more than 30 minutes, then consider relocation to another cloud that may be up to 20% more expensive than the current location.
• If the current SLA for a stack can be met elsewhere with no more than 5 minutes downtime and for 10% discount or better, then move.
• Instantiate two clones of a stack, activate one and connect it to an RDBMS.  Take periodic snapshots of the active stack’s state. If the active stack fails, activate fail-over to standby and send out alerts.
• Support future, scheduled activations. Make cost and SLA based reservations.

Why Autonomic Computing

The advent of distributed computing for achieving cost-efficient performance and availability, coupled with advances in virtualization technologies for further creating cost-efficiencies and making applications more portable, have lead to the burgeoning cloud technologies we are seeing today.  While cloud computing brings the aforementioned benefits of cost efficiency, performance, availability, and portability, it has also come with the price of added complexity, thus creating the need for autonomic computing. Autonomic Computing refers to the self-managing characteristics of distributed computing resources, adapting to unpredictable changes while hiding intrinsic complexity to operators and users

Years back, Amazon started a system called Mechanical Turk, which off-loads simple tasks to humans around the world; people are still working on these tasks, even for pennies. Mostly, it was used for the tasks which can not be performed by computers like describing the content of an image or audio to text translation or other tedious tasks where human intelligence is required. But now, with location-based social networking tools, restaurants, bars, theatres and other services are changing their offerings based on reviews and real-time trends.

This is going to change the nature of work drastically and enable more technologies for collaborative editing. This also allows developers, customers and brand advocates to come together to curate crowd-generated suggestions — leading to more examples of how crowdsourcing will play an important role in marketing, advertising and open innovations. Now non-specialists can show an appetite for participating in an astounding variety of collective, distributed tasks — from the mundane to the most sophisticated.

To tap into the collective intelligence, crowdsourcing is being used to give suggestions to Starbucks, address global climate change problems, and spur NASA’s software development. We’re seeing big brands adopting this approach as well. Unilever dropped its ad agency after 16 years and put up a $10,000 prize for creative marketing ideas, and 50 percent of product initiatives at Procter & Gamble involve significant collaboration with outside innovators.

Gradually, with crowdsourcing, it is becoming easier to tap into the wisdom of the community. Google also runs many crowdsourcing projects internally open only to its employees to forecast product launch dates, new office openings and other strategic corporate decisions. Another output of crowdsourcing is Google Zeitgeist which is based on the aggregation of billions of search queries people typed into Google this year — Zeitgeist captures the spirit of 2010.

The MIT Center for Collective Intelligence has set up many projects to understand how to take advantage of these possibilities. Their basic research question is: “How can people and computers be connected so that collectively they act more intelligently than any individual, groups or computers have ever done before?”

We all know that hyperlinks subvert hierarchy, and the gap in knowledge between the experts and amateurs is shrinking. Crowdsourcing platforms are bringing a renaissance of amateurism that is very useful and crucial to the future of the online marketplace. One day, it may be that the best managed, niche crowds on the Web will become the organization with the most competitive advantage.

Source: IdeaEngineers

It has given me quite a few learning points on how these things work in China

Hosting a website on a server in China:

By definition, all websites hosted on a Chinese server are required to register for an internet content provider (ICP) license. It is basically a permit issued by the Chinese Ministry of Industry and Information Technology that is needed to operate a website in mainland China. The license can be obtained through a simple registration at www.miibeian.gov.cn , where relevant information about the website owner, its content, the hosting provider etc. needs to be provided.

Other useful facts about the ICP license:

-If your website is hosted overseas, you do not need to have an ICP license.

-ICP licenses are not a guarantee that your site won’t be blocked.

-Without an ICP license, your site can be taken down anytime. If hosted in China, your site can be tracked down to its host who can be instructed to take the site down if its registration doesn’t appear in the ICP database.

-Applications done entirely using overseas addresses and contact persons have, to our knowledge, always been denied.

-It is illegal to operate a website in China without an ICP license. Hosting companies are required to shut down websites which do not obtain an ICP license within a certain period of going live, although this is loosely enforced.

Misc. Upshots:

The way internet traffic is handled in China there are other major major constraints you need to take of :

Community has reported serious data inaccuracy in Google Analytics

Even Google says this at http://support.google.com/googleanalytics/bin/answer.py?hl=en&answer=57053

How do i Check Whether My Website Can Be Accessed in China?

http://www.websitepulse.com/help/testtools.china-test.html

http://www.greatfirewallofchina.org/

List of websites blocked in the People’s Republic of China

http://en.wikipedia.org/wiki/List_of_websites_blocked_in_the_People’s_Republic_of_China

CDN in China

http://www.chinacache.com/

http://www.cdnetworks.com/solutions/china-acceleration/

Chinacache claims to be the Akamai of the east http://www.chinacache.com/index.php/news/101-the-akamai-of-the-east.html

Top Cloud/CDN players presence in China:
No Direct presence in China
Nearest Region Presence

Rackspace : Hong Kong

AWS : Singapore region does have an Edge location in Hong Kong for CDN and S3

Akamai :

For further reading:
Gartner’s : How to Improve Application Performance on China’s Internet

What’s not resolved yet?
What’s not resolved yet is what is that effect on internet latency because of the well-known great firewall of China. While media and most of US and EU based organizations blame “Great Firewall” for long internet latency, Gartner does not believe that this is correct. According to community the government does not appear to be consistently examining Internet content, as this appears to be technically impractical. A lot of technical methods are used to block content including IP blocking, DNS filtering, URL filtering, packet filtering etc.

The biggest problem today is that less than 10% of the data created is structured and can be read by currently available computer applications. Most of the information available to an enterprise is highly unstructured – they are documents, emails, content from collaboration tools, intranet and ever-growing wikis. Timely and effective analysis of data can show real competitive advantages, and if managed well, the same data can be used to unlock new sources of many economic values providing fresh insights into solving today’s world problems.

Think about that: if all traffic data in a city is open to both the public and businesses, car rental and car-sharing companies like Zipcar can analyze the data in real-time and make optimal decisions on where they should park their cars. Business organizations are already using the Net Usage Index for Retail, an index provided by Akamai which enables users to monitor the world’s online retail habits 24 x 7, providing real-time insight into the trend of consumers increasing their share of spending on the Web. Data of this magnitude cannot be managed by traditional software platforms, so for vendors, it translates into big opportunities.

Consider India, which is going through a telecom revolution, using consumer trend data to make decisions on pricing and to throw out new consumer plans that will be more successful than previous ones.

Big data has significant potential to create value for both businesses and consumers. It is very interesting to see that most of this growing volume of data is a by-product coming from smartphones, social media sites, millions of sensors and “Internet of Things” type of technologies.  The emergence of real-time location data has also created a new set of opportunities for enterprises in automobile, real estate and causality insurance business.  “Every day I wake up and ask, ‘how can I flow data better, manage data better, analyze data better?” says Rollin Ford, the CIO of Walmart.

With the help of Big Data analysis and management tools, computer algorithms can now crawl through data from a variety of sources, identifying inconsistencies, errors, and fraud. In marketing, Big Data filtrations and analysis help in optimizing recommendation engines to increase online sales and product promotions. In retail, Big Data is used for capturing consumer behavior analysis and then optimizing customer micro-segmentation. It also creates opportunities for innovations in urban planning; for example, Street Bump uses a mobile device’s GPS and accelerometer in tandem to detect, locate and report potholes as you drive in Boston.

“The ability to manage extreme data will be a core competency of enterprises that are increasingly using new forms of information – such as text, social and context – to look for patterns that support business decisions in what we call Pattern-Based Strategy,” said Yvonne Genovese, vice president and distinguished analyst at Gartner.

Today, everyone is a content creator, and the data they produce can hold incredible value. To thrive in this era of Big Data, organizations will ultimately have to rethink the purpose that data holds in driving business values. These trends are just a few among many in the Big Data world which will present tremendous business opportunities for years to come.

For more on the topic, read Christophe Marcant’s white paper on Product Information Management: Definition, Purpose, and Offering, featured on SapientNitro.com.