Cloud Cost Optimization

While working on Cloud, an important step that consideration that needs to be taken care is Cloud Cost Optimization. This requires an approach, strategy and tools to optimize cloud consumption cost. When customer starts the cloud adoption journey there will be strategic drivers and one of the driver is to reduce infrastructure / Operational cost and to increase scalability, performance & agility. Its only possible by adopting right architecture patterns, enabling cloud governance and time to time upgrading/downgrading to right services.  Cost is one of the key pillar in 5 pillars of cloud well-architecture framework and now a days most of the customers are focusing on how to reduce cloud bills and increase application performance without compromising performance of the overall setup. Few points to consider when planning for Cloud cost optimization - 

• How Cloud Pricing Works
• Cloud Cost Optimization Pillars
• Frameworks for Cloud Cost Optimization
• Cost Optimization Strategy, Governance and Best Practices
• Cost Optimization Tools (Cloud-native and 3rd Party Tools) 

Internet of Things (IOT)

With rapid digital transformation and higher connectivity, the Internet of Things is all set to deepen its roots in our industries as well as our lives.  The internet of things (IoT) is a term used to describe the growing network of appliances, homes, and various “smart objects" that communicate with each other to automate and optimize performance. 

The IOT stream involves network specialists who manage connectivity, data analysts who gather data from the devices and interpret it as well as engineers who create the platforms, software, hardware, and systems that allow these devices to function.  

IOT developers focus primarily on the last category which mostly involves  programming software that allows the device at hand to both connect with other systems and function properly on its own. IOT developers are also expected to interface with other devices, create embedded software that’s cloud-compatible and so on.

Simplity

API framework called Simplity can be used for accelerating the web application development for typical transaction processing systems. We would like to introduce the framework to the team.

What is Simplity?

• Rapid Service Development Framework with industry standards supported out of box.
• Infosys supported Open Source project which can help to accelerate the time to market.
• Reusable component Architecture
• Stateless Services
• Reduces development, testing and maintenance effort
• Easy to understand XML based service configuration(No Java code necessary).
• Requires less skill to create and maintain the service.

Best Use cases:

• Service development for Web Client based projects
• Microservice architecture based application
• Data Service development. CRUD Services can be created and reused across multiple services
• API Development. Out of Support of XML and REST support.

Repository: https://github.com/simplity/simplity

Zero Trust Architecture (ZTA)

Recently I came across an interesting architecture pattern called as  Zero Trust Architecture (ZTA).

In traditional ‘perimeter’ based  approach, security models of identity, authentication, and authorization have challenges meeting needs of a IT landscape. There is a need to shift security model to asset-centric or data centric. IT landscape can be expanded using asset & data centric policies and not defining any network boundaries. In the classical approach, we restrict everything to a secure network behind a network firewall/boundary or perimeter whatever you would like to call it. In the zero trust approach, we protect digital assets located/deployed anywhere using a central policy.

Zero Trust - It is an information security approach that focuses on data/information security, including life cycle on any platform or network

Zero Trust Architecture - The implementation of Zero trust security strategy that follows well-defined and assured standards, technical patterns and guidance for organizations

Advantages of ZTA - 

• Enables mobility allowing users to work anywhere using any device
• Improved business confidence, with new security mechanisms to protect data and applications
• Reduced scope of threats to support agility and support complexity

More details on this topic are available on the Open Group website. A whitepaper explaining the concept is available for free download at following URL - https://publications.opengroup.org/w210

Oracle problem - Multiple sessions being opened when querying a table

Recently we were facing an issue in Oracle where SELECT queries on a particular transaction table was opening multiple sessions in Oracle (both 11g and 12c versions).

We checked in multiple environments and using various IDEs/Tools (Toad, SQL Developer etc) and it occurred everywhere.

We also noticed that this issue was happening with only one particular table alone and irrespective of indexes on table or joins in query.

Finally we figured out the reason for this.

The table was created with parallelism option.This is identified by DOP (Degree of Parallelism) setting. For this we have to check 'degree' column in user_table system view against the table. If it shows a value greater than 1, then it means that table has been created with parallelism option ON and Oracle might open multiple sessions when querying the table.

Measuring performance of User Interface and REST service layer separately

Recently we were working on a project where we were looking to separately measure performance of UI (User interface) and REST service layer separately.

Technology involved were- C#.Net 4.5, MVC5.0, JQuery, JSON, XML, Restful Services

So we were searching for a profiling tool which can server our purpose with minimum cost and time/effort involved.

We found several tools that can be used -

• dotTrace
• Mini-Profiler
• MaintainJ
• Dynatrace 

Hope this helps someone who is looking for similar profiling tool.

Few points about Micro-services architecture

Micro-services are not meant to be scaled down version of SOA (Service oriented architecture). They should be based on business context and should not be created based on technical refactoring viewpoint.

Ideal platforms for microservices would be cloud were you can easily scale any one component we want. Example if we notice that one business component (say UI) need support 1000 instances and rest every thing can be 1 or 2 instances then decompose the UI component into a microservice

Cloud platforms and dockers are good area to experiment with micro-services as scaling, and management of  multiple versions can be easily accomplished there.

Few no's for microservices -

• No clustering should be planned
• Try to make integrations independent of programming language (rest, http, xml)
• Avoid single database for multiple microservices

Finally one word of caution, managing multiple microservices is very important. Otherwise after 5 to 6 microservices people loose track of which version is integrated to what.

jQuery data-table with features similar to excel

Recently in one of our projects, we were looking for a jQuery data-table with basic features similar to excel viz. -

• Navigate from cell to cell in a row using right arrow/tab 
• Enable cell edit key press after selecting the cell 
• Enable auto suggest based on the existing data in the column 
• Enable copy of cell and paste to another cell 
• Enable drop-down  to chose values rather than free text 
• Enable column freeze 
• Enable column wise filters

While researching for it, we found few useful components that provides these functionalities. 

1. Slickgrid jQuery plugin
2. Handson table
3. JQGrid

Hope this will help in any teams looking for data-tables with similar functionality.

Best practices for Enterprise level logging

Unless mandated by business requirements, all the log statements should be at minimum logging level. The idea is to reduce the logging overhead as much as possible. Sometimes the overhead is significant to impact application performance.

Use asynchronous logging wherever possible.

Use file rolling and archiving features of the logging framework instead of re-inventing the wheel.

It is a practice among developers to log method entry/exit for troubleshooting. This kind of logging should be done using AOP framework such as Spring AOP, and at debug/trace level. A better practice will be log at only those places where there is a significant processing/turning point.

Do not log any kind of sensitive information such as passwords, credit card numbers, national ids, Personal Identifiable information etc. which could lead to privacy and security issues.

Do not hard-code any credentials (e.g. database credentials when doing database based logging) etc. in logging configuration file.

Do not log huge payloads (such as SOAP request/response etc.) in production but log the important details going out and coming in.

Benchmark the log file growth as per production peak volumes and identify storage requirements (SAN/NAS).

Check regulatory/enterprise guidelines (if any) for the retention period of logs. Device the backup/archival/purge policy accordingly.

Suggest use of log analysis tools such as Splunk or ELK stack for monitoring, troubleshooting, quick searches, dashboards etc.

Secure the log files/sources from external access.

Design a new Framework or Use Existing Framework with a wrapper that can help you capture details better.

Make sure the exception or event is logged only at source. Most of the times, every component participating, in the process, logs the exception/event . This not only leads to unnecessary logging but also sometimes logs incorrect information for tracing the issue.

Have a unique event ID(within application & combination with an application id should make it unique across applications) for every Exception/Process/Event so the Support Group can triage, without much hassle.

Test SMTP server for DEV/Testing

Many times we face problem testing email settings/code in our development environment if we don't have a SMTP server in our development environment.

There is a easy solution for it.

Use dummy SMTP server - SMTP4DEV available in CodePlex at following location -

http://smtp4dev.codeplex.com/

Once it is launched, it sits in your system tray and listens on port no. 25 of your localhost (127.0.0.1) server. This helps in receiving and viewing messages and verifying their format/structure.

It however cannot be used for sending messages.

Detailed instructions for installing and using this nifty tool can be found at its CodePlex location.

MSDN Subscriptions is now Visual Studio Subscriptions

From Oct 4, 2016, MSDN subscriptions will be re-christened as Visual Studio subscriptions.

After 4 October 2016, you need to go to the new Visual Studio Subscriptions portal to access your subscription.

Please note that you will have no access to the current MSDN Subscriptions portal between 9:00 pm (PST) 3 October and 9:00 pm (PST) 4 October, while we complete the migration.

In case of any questions, check the FAQ or contact Visual Studio Customer Service

Tech Trivia

• The word 'Robot' comes from Czech word 'ROBOTA'. In English, it translates to 'forced labor'. 
• A Red Panda is an animal native to the Himalayas and Southwestern China. The English word for Red Panda is Firefox which is where the browser gets its name from. So the Firefox Logo is actually a Red Panda and not Fox as thought by many (including me!).
• Android is Andy Rubin who is the co-creator of Android. It was a nick-name given to him at Apple before he joined Google, for his obsession and love for robots.
• Creators of PNG file format wanted it to be pronounced as PING.
• Nothing can be compared to the human mind. If a computer is designed which is as powerful as the human brain, then it would be able to perform about 38 thousand trillion operations per second. Also, it will be able to hold about 3584 terabytes of memory.
• HP, Microsoft & Apple have one not so obvious thing in common - they all started in a garage.
• If you find a security bug in Facebook, they are willing to pay money ($500 and upwards) for you to tell them about it.
• 160 billion emails are sent daily, 97% of which are spam.
• Wikipedia uses an army of anti-vadal bots. These bots prevent vandalism of wiki pages. As per them, vandalism is anyone signing up and editing pages with an intent to destroy or alter content in a negative manner.
• Ubuntu is one of the most popular distribution channels of Linux. The word Ubuntu comes  from the African word meaning 'I am because of you'.
• The actual name of Android mascot is 'Bugdroid', though it is not official, the team at Google refer to it by this name.
• The first known cell phone virus, Cabir.A, appeared in 2004.

C# - Serialize Object to XML HttpResponseMessage

Sometimes we have an explicit requirement to serialize a C# object to XML HttpResponseMessage format.

This generic function can be used to achieve this conversion -

public static HttpResponseMessage CreateXmlResponse<T>(T objectToSerialize)

{

    return new HttpResponseMessage(HttpStatusCode.OK)

    {

        Content = new ObjectContent<T>(objectToSerialize,

                                        new System.Net.Http.Formatting.XmlMediaTypeFormatter

                                        {

                                            UseXmlSerializer = true

                                        })

    };

}

The function takes an object of type 'T' as input and serializes it using XMLMediaTypeFormatter XML serializer available in .NET namespace System.Net.Http.Formatting.

A sample function call be like this - 

var response = CreateXmlResponse<ObjectClass>(object);

C# - Convert JSON string to JSON HttpResponseMessage

Use following method/function to convert string having JSON data to JSON HttpResponseMessage -

public static HttpResponseMessage CreateJsonResponse(string json)

{

    return new HttpResponseMessage()

    {

        Content = new StringContent(json, System.Text.Encoding.UTF8, "application/json")

    };

}

Following code can be used to call this method/function

var response = CreateJsonResponse(jSonString);

Following code can be used to call this method/function

Convert time-stamp in UTC format to local date time

Some times it is required to convert a time-stamp in UTC format to your local system data time.

A UTC time-stamp format looks something like this - 2016-08-23T11:59:00Z

This time-stamp is in UTC zone and following C# code can be used to convert this into local date time. As an example, here I am converting it to 'Central Standard Time (CST)'. Depending on your time zone, you can pass a different value.

public static string GetLocalDateTime(string timeStamp)

{

    TimeZoneInfo cstZone = TimeZoneInfo.FindSystemTimeZoneById("Central Standard Time");

    DateTime cstTime = TimeZoneInfo.ConvertTimeFromUtc(Convert.ToDateTime(timeStamp), 

                                                       cstZone);

    return cstTime.ToString();

}

An example call will be - 

var localDateTime = GetLocalDateTime("2016-08-23T11:59:00Z");

Output will be -  8/23/2016 6:59:00 AM

Options for SMS solution

One of our clients had the following requirements around their SMS solution:-

1.       SMS Authentication
2.       SMS Sales (Ordering pay per view / adding balance)
3.       Alerts and notifications

They were currently using a messaging platform which was very specific to a particular country. To reduce the overall cost, they were looking at other solutions and wanted to make use of the open source solutions in this space to replace the their existing solution. They were interested in building a custom solution and paying to the network carrier for billing rather than using a 3rd party off the shelf solution.

We performed study and comparison of various open source and COTS SMS Gateways. Found this article useful in comparison -

http://socialcompare.com/en/comparison/sms-gateways-1odvv6xv

The final solution recommended was Kannel. It is one of the highly scalable and efficient open source SMS and WAP Gateway solution that majority of the Messaging Providers uses in their suite. It is based on C technology and supports connectivity to various SMS Centers. It also provides HTTP based API for integration with other business suites for sending and receiving messages.

Hope this helps someone who is looking for similar comparison and solution.

The Open Group Paris - Oct 24-27, 2016 - Early Bird Rate Ends Sept 16

The theme for The Open Group Paris event is e-Government. Attendees will benefit from the opportunity to learn from industry leaders, network with peers and explore content that is relevant to both themselves and their organizations.

Emphasis will be on how techniques (such as Enterprise Architecture and Business Design) and how standards (such as TOGAF® and ArchiMate®) are acting as a foundational core for enterprise transformation.

Topics include: Issues surrounding business transformation, EA in government, digital business and customer experience, IT4IT™ in Practice, professional development and cyber-security.

Early Bird Registration - Ends on Friday, September 16, 2016

Register Now

Webinar: The TOGAF® Framework and Accelerated Delivery

Date: September 9, 2016 - 3:00pm (BST), 10:00am (EDT), 7:00am (PDT)
Speaker: Alan Simmonds (Consultant, Good e-Learning)

This webinar will look into accelerated delivery (not just Agile!) and how we can start to integrate this thinking across the Enterprise Architecture development cycle. TOGAF®, an Open Group standard, is a large framework and what this provides is a number of techniques for us to use and adapt for the accelerated delivery approach.

We’ll look at what accelerated delivery means, whether it is relevant in the Enterprise Architecture world and how we can start to integrate some of this thinking into our own architecture work.

To register, click here

The Open Group Event @ Austin (July 18-21, 2016)

When it comes to Enterprise Architectures, what does it mean to be "open"? In the technology industry, often we hear about things such as "open standards" or "open source" but what does open mean in each context and why are those differences important?

This event will focus on how organizations can use openness as an advantage and how the use of both open standards and open source can help enterprises support their digital business strategies. Sessions will look at the opportunities, advantages, risks and challenges of openness within organizations.

Keynote sessions will include:

The Future of Business Architecture: Challenges and Opportunities (Jeff Scott - President - Business Innovation Partners)

ArchiMate® 3.0 A New Standard for Architecture (Iver Band - Enterprise Architect - Cambia Health Solutions, Mark Lankhorst - Managing Consultant and Service Line Manager - BIZZdesign)

Driving IT Strategic Planning at ExxonMobil with IT4ITTM (Rick Solis, IT Business Architect - ExxonMobil Global Services Co. )

For details regarding fees, agenda, other information and registration, please use following URL

SharePoint 2013 - Opening Search results in new tab/window

In SharePoint 2013, we can use Search Center site to show/display SharePoint search results.

Each search result has 3 parts -

• Heading (which acts as URL/link)
• Description
• Actual URL

We can click on the heading which is actually a hyperlink to go to the search result destination. By default, the search result is opened in same window if we click on hyperlink. If you would like to open the search result in a new window, then it can be achieved by customizing the search results display template. Here are the steps for same - 

• Go to Site Settings (You should have Site Collection Administrator permission).
• Go to section Web Designer Galleries.
• Click on Master pages and page layouts.
• A document library will open. Go to folder Display Templates --> Search
• Find following file in the folder - Item_CommonItem_Body.html
• Check-out the file and edit in any text editor.
• Find following line of text - 

var titleHtml = String.format('<a clicktype="{0}" id="{1}" href="{2}" class="ms-srch-item-link" title="{3}" onfocus="{4}" {5} {6}</a>',

Add target="_blank" between {5} and {6}. After modification, it will look like this

var titleHtml = String.format('<a clicktype="{0}" id="{1}" href="{2}" class="ms-srch-item-link" title="{3}" onfocus="{4}" {5} target="_blank">{6}</a>',

• Save the file and upload it back to same location from where it was downloaded.
• Check-in and Publish the file.
• Make sure to publish as a major version. In the past, we have faced problem in this template getting applied to other users without major version being published.

Perform a search on the search center site and click on any of the search results. The hyperlinks will now open in new tab/window.

Hope this helps !!!