DataInsighter specializes in building end-to-end systems that unlock the potential hidden in data. Using advanced analytics and machine learning to combine the power of Cloud, Mobile and Internet of Things (IoT), we build systems that transform data into valuable insights, delivering this power whenever and wherever it is needed. Through strategic consulting, as well as top industry-level architecture, development and integration services, we help businesses quickly and efficiently discover new insights to gain competitive advantage.
Who We Are
Headquartered in Larnaca, Cyprus, with offices in Houston, Texas and Redmond, Washington, USA, DataInsighter is a group of world-class engineers and data scientists with decades of industry experience. Focusing on the three cutting-edge areas of Cloud, Mobile and Internet of Things, we create unique end-to-end solutions that combine these components with rich Analytics to empower organisations through data. Our history as a company is grounded in industrial-strength software development, systems architecture and data analysis in and out of the cloud, including Microsoft’s core Analytics and Business Intelligence team and Microsoft’s Windows Data Science group. With extensive knowledge in developing scalable, high-performance web services, building applications that run on multiple platforms including Windows, iOS and Android, integrating existing and new systems, and providing data analytics, visualizations, statistical modeling and machine learning on top of all components, we have a passion for building systems, understanding data and unlocking insights that can make a difference whenever and wherever they are needed.
What We Do
DataInsighter provides both consulting and development services for any organization looking to take their data and systems to the next level. As a group of world-class engineers, we know how to build cutting-edge, industrial-strength software systems. Using agile development methodologies, we apply our years of experience to take requirements and then satisfy them in the most efficient way possible. Specializing in, but not limited to, Microsoft’s powerful suite of technologies and services, we know how to deal with challenging constraints and tailor our approach to each problem. Building brand new systems or integrating with legacy assets, we find the best balance of cutting-edge technologies and minimal rework to get organizations where they want to be as quickly as possible. At every step of the way, we focus on building a view into the data being produced and consumed by the system, looking for opportunities to go beyond traditional Business Intelligence, into the world of predictive analytics and machine learning.
More Than Just Development
When approaching any new challenge, we apply a process that lets us build the most useful system in the fastest but most stable way possible. The roadmap generated by this process is extremely useful to companies and so we offer it as a service separate from development. The process begins by analyzing an operation and identifying key technological opportunities. Delving into increasing levels of detail, we then design the most appropriate architecture and lay out a path from the current state to the final goal, with iterative milestones that provide value quickly and allow for maximum flexibility. To ensure that a plan is fully fleshed out, we can build smaller-scale, prototype systems as proof of concept.
Whether your organization is simply looking for ways to improve your data and systems or advice on how to take advantage of the latest in Cloud, Mobile, IoT or Data Analytics, we can help. Beyond the theoretical, we know how to make the magic happen, taking any organization from data curious to data-driven.
DATA AND TECHNOLOGY
In the fast-paced world of evolving technologies, it is very difficult to keep track of the latest trends and anticipate future needs. With extensive experience in building high-tech solutions, DataInsighter’s technical experts can help an organization meet current and future goals through strategic consulting.
When developing a technology strategy, we start with an analysis of operations and current systems, focusing on pain-points and future goals. From this, we construct a high-level recommendation of areas of opportunity to transform the organization through web services, cloud computing, mobile and analytics. This stage is a mixture of brain-storming and taking stock, balancing ambitious goals with real-world constraints. Almost anything is possible with intelligent planning and efficient use of time.
We then dig deeper into the problem, developing a detailed architecture, built with flexibility, testability, maintainability and performance firmly in mind. At Microsoft, our engineers gained experience building products that are used by millions of people around the world. This experience means we are ready to tackle challenging technical problems by creating the most appropriate architecture and implementing an efficient, high quality solution.
Armed with the final goal, we then develop a technology roadmap that outlines a path toward the full system, with intermediate milestones that allow real impact sooner rather than later and giving organizations the option to choose a full or partial solution.
Finding potential pitfalls in a system early in the development cycle is critical. The best way to achieve this is to build a trimmed down proof-of-concept in a short period of time to evaluate design decisions and test assumptions. Our team specializes in focusing on the critical business and technical risks and builds systems to quickly validate the proposed solution.
DataInsighter knows how to help organizations at any level of technical maturity by fixing immediate problems with an eye to the future, where they can utilize the power of the Cloud, Mobile Apps and IoT, providing analytics to make decisions based on data rather than a feelings or assumptions.
Increasingly, cloud computing is becoming a major component of the technology and business strategy for companies of all sizes. Smart companies are quickly adapting to this change and moving computing resources to the cloud. This shift lowers cost by reducing the need to maintain infrastructure and it increases competitive edge by enabling organizations to take advantage of potentially unlimited computational and storage resources available in the cloud. Cloud computing provides a way to pay only for what is needed, when it is needed, rather than trying to forecast capacity. Furthermore, the move enables greater collaboration and mobility for workers who no longer need to be tied to their office desk to access and work with company data and applications.
Should all data be in the cloud?
This is a common question and a common issue faced by many companies. Public cloud resources are extremely useful as their potential capacity is essentially limitless. The provider is responsible for planning, building and maintaining the infrastructure. This capacity means that storage requirements are not a concern, and the processing capacity required to crunch this data is available when needed. However, there are very real concerns for based on legacy systems as well as security and privacy of key data assets. There are a number of different options here, depending on the particular needs and constraints of the organization.
One solution is to use what is known as a Private Cloud. This is a cloud instance completely separate from any other cloud users, giving total control to the owner. While generally more complex, it allows for the same interface and seamless integration with the public cloud, but with full control and ownership of the data and processing.
A second, very powerful option is to combine the power of public and private cloud to form a Hybrid Cloud. This allows a company to host their own private cloud, with all of the benefits and control it provides, but still make use of the lower cost and complexity of the public cloud for less sensitive data and operations. Work can be performed in public or private environments, or work may be performed to anonymize data for processing on the public side. Jobs and reports use data where and when they are needed, with the appropriate level of control.
The cloud is a fantastic tool and its use will increase dramatically over time, but for many reasons – such as performance, existing assets, legacy procedures, privacy and security – not all data and computing needs to be, or should be, in the cloud. Traditional relational databases and data warehouses are still critical to any business and the most ideal solution will be a mixture of cloud and on-premises computing. Modern database management systems and data warehouses are fully equipped to bridge the gap between relational data (i.e. traditional) and non-relational (i.e. Big Data) data, providing a means to store and process data wherever it makes the most sense, in or out of the cloud.
Moving to the Cloud
We offer a wide range of services including:
- Custom cloud application development
- Application modernization
- Integration with cloud services
- Cloud application prototyping
DataInsighter’s engineers are highly trained and experienced with cloud technologies and how to best take advantage of them along with other, traditional data and computing resources. We begin any potential cloud-based project by working to understand the underlying business requirements and then formulate a detailed development plan, taking into consideration performance, scalability, security, integration and deployment. Ensuring that the latest frameworks and tools are used, we implement the solution using Agile software engineering practices that allow for progress to be monitored along the way.
If you’re ready to take advantage of the cloud, or if you’re just curious to learn more about the potential of a cloud-based solution, contact us to discuss your unique business needs and find out how DataInsighter can help take your organization to the next level.
INTERNET OF THINGS
The Internet of Things, or IoT, refers to the growing number of devices connected to the internet – from mobile phones to thermostats, temperature sensors to GPS trackers. More specifically, it refers to the network these devices create and the abundance of data sent over this network. The power of IoT lies in this data and the combination of IoT and cloud computing presents an opportunity to transform a number of industries by making them smarter. Smart Energy systems allow devices in a power network to be monitored, flagging excessive usage and allowing for efficient distribution of load. Smart Cities are using IoT devices to monitor traffic, pedestrian movement and environmental conditions to improve planning, assign resources and aid emergency services. Smart Manufacturing allows for an entire plant to be monitored to improve throughput, identify bottlenecks and optimize the entire production line. DataInsighter specializes in providing a platform to take all of this IoT data and put it to work through collection, cleansing, analysis and access to the data and insights wherever they are needed.
The traditional approach to monitoring and control of a system is through proprietary devices. These devices are generally heavyweight, designed for predefined installations that may not meet the requirements of the network. Their communication and data protocols are also proprietary and inflexible, making it difficult to get the most out of what is being measured or controlled and limiting the ability to change the network over time by adding devices or changing device vendors. IoT solves these problems by providing for both scalability and flexibility. Scalability is a result of the fact that an IoT device simply needs to connect to the internet and transmit data. Any device that can take a measurement and transmit to an appropriate web-based backend can easily extend the network. In the same manner, a change in client device manufacturer is invisible to the IoT platform, so long as it is sending the data with the appropriate web call, meaning an extra level of flexibility. This means that the number of IoT devices in a system is dynamic – no need to anticipate all future needs, just implement what is immediately required and then scale up or down as requirements change in the future by adding whatever device makes sense. From monitoring the power usage of a residential home or business, all the way up to a city full of traffic and pedestrians, IoT provides extra freedom to adapt over time.
The real power of an IoT solution lies in the data and taking advantage a full suite of data storage and analytics provides a unique opportunity for Business Intelligence as well as advanced analytics through the use of machine learning and statistical modeling. Once an IoT system is instrumented and reporting its state, the entire network can be modeled and visualized. Broadly, there are two types of analytics to consider: real-time and latent (post processing).
Live Monitoring and Alerting
Real-time analytics allow for constant updating of KPIs and system health measures for live monitoring and alerting. Cloud-based IoT platforms are designed to ingest millions of events per second, correlate across related data sources and push information to applications where it is needed, in near real-time. Single devices do not need to be monitored in isolation, and their data can be combined centrally to give a more informed overview of an entire network. Dashboards and applications can be built, with smart, data-driven indicators when there is a problem at any point in the system.
Post-Processing for Structure and Historical Analysis
Latent analytics, or post-processing, represent a unique opportunity to analyze data beyond the obvious descriptive power. As events are received in real-time, they are forwarded to appropriate reporting and alerting mechanisms, before being stored for later processing. Using big data technologies such as Hadoop, processing jobs are run on any necessary data, scaling elastically to parallelize the processing and process any amount of data. This is a mixture of scheduled and adhoc jobs, with scheduled jobs producing processed (cooked) data that can be stored in traditional data warehouses for faster reporting and analysis.
By providing a data-driven view of the entire process, all decisions can be made using data – no more ‘guessing’ required. This in itself is huge, but the benefits do not stop there.
Using Machine Learning, also known as Predictive Modeling, problems in the system can be detected before they happen, leading to predictive maintenance. Since each piece of equipment is fully instrumented, the signs of an issue occurring can be found in the data being produced. Models may be prototyped and experimented with before being deployed in the production system to report predictions and track problems before they occur.
Instrumenting the process allows the entire system to be optimized through data. Experimenting with tweaks to single components can have their impact fully measured across the whole deployment. The data output from any component can be fed back into the system, providing a rich feedback loop that goes beyond a traditional setup.
COMBINING DATA SOURCES
Instrumentation and measurement need not be limited to the IoT network itself – this data can be integrated with other organizational data sources to boost efficiency through holistic insights. As an example, supply chain data in a Smart Manufacturing setup can form part of the control feedback loop, potentially throttling production capacity in reaction to supply shortage. Moving further along, Customer Relationship Management (CRM) data may also be fully integrated with Sales, Marketing and Social data. Sales teams can view manufacturing output data while on the road and report directly to customers. This full suite of data can be leveraged for improved resource management and planning.
Overall, the combination of IoT devices and cloud computing matches the capability of existing systems, while providing extra flexibility and lower costs. Beyond current capabilities, an IoT solution allows for a data-driven view of the entire system, implementation of new, powerful control algorithms and the ability to apply advanced data analysis and statistical modeling techniques on a centralized repository of data.
Most organizations have existing business systems and associated data that were never designed to work together. However, this functionality – and, perhaps more importantly, this data – is far more useful and powerful when it may be managed and used collectively, wherever and whenever it is needed. Additionally, the prevalence of mobile devices and telecommuters has led not just to a request for common access, but even to an expectation of availability of company services and data regardless of location.
Solving this problem is challenging, both from an architectural and development perspective. The best approach is to follow a set of established industry patterns, proven to work in a number of different situations. At DataInsighter, we have experience with many of these different patterns and can ensure that data (and functionality) is available where it is needed.
A fairly recent concept, still lacking a formal definition, the use of Web APIs allows an organization to expose their data and systems internally and externally using lightweight, broadly-supported web calls. This approach allows for minimal processing requirements on the client-side, meaning that mobile browsers and smartphone apps can easily take advantage. It also provides the ability to expose services in a common way to internal or external clients and monitor and govern their usage through API management.
The use of Web APIs also allows the ability to expose systems via the cloud, pushing system access control off premises. This is a perfect use-case for the cloud, allowing the use of the API to scale effortlessly and simple integration with systems as they are also moved to the cloud. Spikes in API usage are handled smoothly, minimizing potential outages, while the rich tooling and support from cloud providers allows for rigorous API governance.
Even with the prevalence of Web APIs, they are not always appropriate and can only be built on top of enterprise systems that are ready to support the pattern. In many cases, the use of more traditional service-oriented architecture (SOA) integration is necessary, along with writing custom adapters for new and existing systems to communicate directly to each other. Using the latest techniques has a lot of advantages, but it is critical to find the right approach regardless of the particular technology used to achieve it.
Our engineers are familiar with all of the complexities an integration project involves and work to find the right solution that strikes the perfect balance of custom work, extensibility, performance and security. We work to build the right solution and make data and functionality available wherever it is needed.
DataInsighter engineers and data scientists employ a wide range of tools and technologies, adapting to the needs of each data challenge.
IN THE CLOUD
Cloud computing offers the ability to scale storage and computation based on necessity rather than being limited by an organization’s computing capacity. This means that data can be acquired and stored indefinitely using services such as Amazon AWS and EC2 as well as SQL and Windows Azure. At DataInsighter, we make full use of the latest NoSQL technologies by running Hadoop jobs with Amazon Elastic Map Reduce or Microsoft HDInsight.
We understand that not all data is suitable for deployment to a public cloud service. Existing infrastructures may not be easy to migrate and some data may have security or privacy requirements excluding it from a public service. In these instances we employ the Microsoft BI stack for on-premises data management and analysis including SQL Server Analysis and Reporting Services with integration into Microsoft SharePoint.
MACHINE LEARNING AND RAPID PROTOTYPING
Data at scale is inherently difficult to analyze. To get the most out of data it is important to use state-of-the-art statistical packages such as R, Python and Matlab. We sample huge collections of data to gather a representative set that is much smaller and easier to handle. We then rapidly prototype different analytical approaches including trained and untrained machine learning techniques. By using this fast, iterative process, we can quickly draw conclusions about effective models and then plan a scale-out of the most effective Big Data solution.
Often the biggest impact data can have is through visualizations. After the data has been stored, cleaned and analyzed, we can distill the most important parts into Microsoft Excel spreadsheets where we will create appropriate tables, charts and graphics to make the insights obvious and useful. We have experience sharing and hosting these through Microsoft SharePoint and PowerPoint and embedding graphics into web pages.
CUSTOM SOFTWARE AND SYSTEMS
With our years of combined experience in software development, we can identify pieces of the system that may be automated or scaled out using custom .NET, Java or C++ code. Whether it is a sensor producing isolated data that should be consolidated, or a robust web service to host the data and insights produced during analysis, we can build a tailored solution that adds to the overall value of the system.
©2018 DATAINSIGHTER. ALL RIGHTS RESERVED.