Cloud Cost Optimization

While working on Cloud, an important step that consideration that needs to be taken care is Cloud Cost Optimization. This requires an approach, strategy and tools to optimize cloud consumption cost. When customer starts the cloud adoption journey there will be strategic drivers and one of the driver is to reduce infrastructure / Operational cost and to increase scalability, performance & agility. Its only possible by adopting right architecture patterns, enabling cloud governance and time to time upgrading/downgrading to right services.  Cost is one of the key pillar in 5 pillars of cloud well-architecture framework and now a days most of the customers are focusing on how to reduce cloud bills and increase application performance without compromising performance of the overall setup. Few points to consider when planning for Cloud cost optimization - 

• How Cloud Pricing Works
• Cloud Cost Optimization Pillars
• Frameworks for Cloud Cost Optimization
• Cost Optimization Strategy, Governance and Best Practices
• Cost Optimization Tools (Cloud-native and 3rd Party Tools) 

Internet of Things (IOT)

With rapid digital transformation and higher connectivity, the Internet of Things is all set to deepen its roots in our industries as well as our lives.  The internet of things (IoT) is a term used to describe the growing network of appliances, homes, and various “smart objects" that communicate with each other to automate and optimize performance. 

The IOT stream involves network specialists who manage connectivity, data analysts who gather data from the devices and interpret it as well as engineers who create the platforms, software, hardware, and systems that allow these devices to function.  

IOT developers focus primarily on the last category which mostly involves  programming software that allows the device at hand to both connect with other systems and function properly on its own. IOT developers are also expected to interface with other devices, create embedded software that’s cloud-compatible and so on.

Simplity

API framework called Simplity can be used for accelerating the web application development for typical transaction processing systems. We would like to introduce the framework to the team.

What is Simplity?

• Rapid Service Development Framework with industry standards supported out of box.
• Infosys supported Open Source project which can help to accelerate the time to market.
• Reusable component Architecture
• Stateless Services
• Reduces development, testing and maintenance effort
• Easy to understand XML based service configuration(No Java code necessary).
• Requires less skill to create and maintain the service.

Best Use cases:

• Service development for Web Client based projects
• Microservice architecture based application
• Data Service development. CRUD Services can be created and reused across multiple services
• API Development. Out of Support of XML and REST support.

Repository: https://github.com/simplity/simplity

Zero Trust Architecture (ZTA)

Recently I came across an interesting architecture pattern called as  Zero Trust Architecture (ZTA).

In traditional ‘perimeter’ based  approach, security models of identity, authentication, and authorization have challenges meeting needs of a IT landscape. There is a need to shift security model to asset-centric or data centric. IT landscape can be expanded using asset & data centric policies and not defining any network boundaries. In the classical approach, we restrict everything to a secure network behind a network firewall/boundary or perimeter whatever you would like to call it. In the zero trust approach, we protect digital assets located/deployed anywhere using a central policy.

Zero Trust - It is an information security approach that focuses on data/information security, including life cycle on any platform or network

Zero Trust Architecture - The implementation of Zero trust security strategy that follows well-defined and assured standards, technical patterns and guidance for organizations

Advantages of ZTA - 

• Enables mobility allowing users to work anywhere using any device
• Improved business confidence, with new security mechanisms to protect data and applications
• Reduced scope of threats to support agility and support complexity

More details on this topic are available on the Open Group website. A whitepaper explaining the concept is available for free download at following URL - https://publications.opengroup.org/w210

Oracle problem - Multiple sessions being opened when querying a table

Recently we were facing an issue in Oracle where SELECT queries on a particular transaction table was opening multiple sessions in Oracle (both 11g and 12c versions).

We checked in multiple environments and using various IDEs/Tools (Toad, SQL Developer etc) and it occurred everywhere.

We also noticed that this issue was happening with only one particular table alone and irrespective of indexes on table or joins in query.

Finally we figured out the reason for this.

The table was created with parallelism option.This is identified by DOP (Degree of Parallelism) setting. For this we have to check 'degree' column in user_table system view against the table. If it shows a value greater than 1, then it means that table has been created with parallelism option ON and Oracle might open multiple sessions when querying the table.

Measuring performance of User Interface and REST service layer separately

Recently we were working on a project where we were looking to separately measure performance of UI (User interface) and REST service layer separately.

Technology involved were- C#.Net 4.5, MVC5.0, JQuery, JSON, XML, Restful Services

So we were searching for a profiling tool which can server our purpose with minimum cost and time/effort involved.

We found several tools that can be used -

• dotTrace
• Mini-Profiler
• MaintainJ
• Dynatrace 

Hope this helps someone who is looking for similar profiling tool.

Few points about Micro-services architecture

Micro-services are not meant to be scaled down version of SOA (Service oriented architecture). They should be based on business context and should not be created based on technical refactoring viewpoint.

Ideal platforms for microservices would be cloud were you can easily scale any one component we want. Example if we notice that one business component (say UI) need support 1000 instances and rest every thing can be 1 or 2 instances then decompose the UI component into a microservice

Cloud platforms and dockers are good area to experiment with micro-services as scaling, and management of  multiple versions can be easily accomplished there.

Few no's for microservices -

• No clustering should be planned
• Try to make integrations independent of programming language (rest, http, xml)
• Avoid single database for multiple microservices

Finally one word of caution, managing multiple microservices is very important. Otherwise after 5 to 6 microservices people loose track of which version is integrated to what.

jQuery data-table with features similar to excel

Recently in one of our projects, we were looking for a jQuery data-table with basic features similar to excel viz. -

• Navigate from cell to cell in a row using right arrow/tab 
• Enable cell edit key press after selecting the cell 
• Enable auto suggest based on the existing data in the column 
• Enable copy of cell and paste to another cell 
• Enable drop-down  to chose values rather than free text 
• Enable column freeze 
• Enable column wise filters

While researching for it, we found few useful components that provides these functionalities. 

1. Slickgrid jQuery plugin
2. Handson table
3. JQGrid

Hope this will help in any teams looking for data-tables with similar functionality.

Best practices for Enterprise level logging

Unless mandated by business requirements, all the log statements should be at minimum logging level. The idea is to reduce the logging overhead as much as possible. Sometimes the overhead is significant to impact application performance.

Use asynchronous logging wherever possible.

Use file rolling and archiving features of the logging framework instead of re-inventing the wheel.

It is a practice among developers to log method entry/exit for troubleshooting. This kind of logging should be done using AOP framework such as Spring AOP, and at debug/trace level. A better practice will be log at only those places where there is a significant processing/turning point.

Do not log any kind of sensitive information such as passwords, credit card numbers, national ids, Personal Identifiable information etc. which could lead to privacy and security issues.

Do not hard-code any credentials (e.g. database credentials when doing database based logging) etc. in logging configuration file.

Do not log huge payloads (such as SOAP request/response etc.) in production but log the important details going out and coming in.

Benchmark the log file growth as per production peak volumes and identify storage requirements (SAN/NAS).

Check regulatory/enterprise guidelines (if any) for the retention period of logs. Device the backup/archival/purge policy accordingly.

Suggest use of log analysis tools such as Splunk or ELK stack for monitoring, troubleshooting, quick searches, dashboards etc.

Secure the log files/sources from external access.

Design a new Framework or Use Existing Framework with a wrapper that can help you capture details better.

Make sure the exception or event is logged only at source. Most of the times, every component participating, in the process, logs the exception/event . This not only leads to unnecessary logging but also sometimes logs incorrect information for tracing the issue.

Have a unique event ID(within application & combination with an application id should make it unique across applications) for every Exception/Process/Event so the Support Group can triage, without much hassle.

Test SMTP server for DEV/Testing

Many times we face problem testing email settings/code in our development environment if we don't have a SMTP server in our development environment.

There is a easy solution for it.

Use dummy SMTP server - SMTP4DEV available in CodePlex at following location -

http://smtp4dev.codeplex.com/

Once it is launched, it sits in your system tray and listens on port no. 25 of your localhost (127.0.0.1) server. This helps in receiving and viewing messages and verifying their format/structure.

It however cannot be used for sending messages.

Detailed instructions for installing and using this nifty tool can be found at its CodePlex location.