Material Big Data

Lanzados ppts informativos de tecnologías BigData: Hadoop, Hbase, Hive, Zookeeper...

Apuntate al Curso gratuito para aprender Business Intelligence. Plazas limitadas!!

Diseño multidimensional, OLAP, ETL, visualización, open source...

Pentaho Analytics. Un gran salto

Ya se ha lanzado Pentaho 7 y con grandes sorpresas. Descubre con nosotros las mejoras de la mejor suite Open BI

La mejor oferta de Cusos Open Source

Después de la gran acogida de nuestros Cursos Open Source, eminentemente prácticos, lanzamos las convocatorias de 2017

18 oct. 2017

Human Resources Analytics


Human Resources LinceBI Analytics solution is based on open source including KPIs, Reports, OLAP Analysis, Dashboards, Scorecards, Big Data and Machine Learning with 'predefined templates, dashboards and KPIs/ratios and fully customizable environment

Manage budgets efficiently and maximize revenues and costs in favour of collective benefit.



Do more with less! Through innovative techniques of Data Mining and Social Intelligence to maximize objectives, identifying trends related to workers behavior and satisfaction in order to answer their demands efficiently and improve engagement










15 oct. 2017

Comparativa de Costes Tableau vs PowerBI

 

Os dejamos un documento listo para descargar, con una comparativa muy completa de costes entre Tableau y PowerBI (hay que decir que el informe ha sido encargado por Tableau, por lo que puede tener cierto sesgo). 

Por ejemplo, en cuanto al esfuerzo de este tipo de proyectos, si tenemos en cuenta que ambas son herramientas de Data Discovery (usuario final), no se tiene suficientemente en cuenta la parte más importante, el modelado, ETL, Data Quality, etc... 

En la práctica, estas herramientas, necesitan también de herramientas ETL, metadatos, MDM, Data Quality que garanticen la correcta implementación en entornos en producción

Para una comparativa de funcionalidades técnicas echad un vistazo a la Comparativa de herramientas Business Intelligence

Ver también: Como preparar un entorno Big Data OLAP con Tableau y con PowerBI








Use Case “Dashboard with Kylin (OLAP Hadoop) & Power BI”



In recent posts, we explained how to fill the gap between Big Data and OLAP, using Tableau, Pentaho and Apache Zeppelin.

Now, we´ll show you how to use PowerBI for Big Data Dashboards using Apache Kylin. Also try online in our Big Data Demo site


Arquitecture:
In this use case we have used together Apache Kylin and Power BI to support interactive data analysis (OLAP) and developing a dashboard, from data source with Big Data features (Volume, Speed, Variety).


The data source contains the last 15 years of academic data from a big university. Over this data source, we have designed a multidimensional model with the aim of analyze student’s academic performance. We have stored in our Data Warehouse about 100 million rows, with metrics like credits, passed subjects, etc. The analysis of these facts is based on dimensions like gender, qualification, date, time or academic year.
However this data volume is too large to be analyzed using traditional database systems for OLAP interactive analysis. To address this issue, we decide to try Apache Kylin, a new technology that promises sub second interactive queries for data Volumes over billions and trillion of rows on the fact table.
Apache Kylin architecture is based on two Hadoop stack technologies: Apache Hive and HBase. First, we have to implement the Data Warehouse (DW) on Hive database using a star or a snow flake schemas. Once we have implemented one of these data models, we can define an OLAP cube on Kylin. 
To this end, we have also to define a Kylin’s cube model using Kylin’s GUI with wizard. At this moment, Kylin can generate the MOLAP cube in an automatic process. After cube creation, we can query the OLAP cube using SQL queries or connecting to a BI tool using the available J/ODBC connectors.
With aim to explore the data and generate visualizations that allows users to extract useful knowledge from data, we have chosen Microsoft Power BI tools: Power BI Desktop and Power BI Service (free of charge version).
Power BI Desktop is a completely free desktop self-service BI tool that enable users to create professional dashboards easily, dragging and dropping data concepts and charts to a new dashboard. Using this tool we have developed a dashboard, similar to our use cases with Tableau or Apache Zeppelin.
Once designed the dashboard, we have published it on the Web with Power BI cloud service (free edition). In other to do that, we have to create an extract of the data and upload it with the dashboard.  This process is transparent to users, who also can program data refreshing frequency using Pro or Premium versions of the Power BI service (commercial tools).


Apache Kylin:


Developed by eBay and later released as Apache Open Source Project, Kylin is an open source analytical middle ware that supports the support analysis OLAP of big volumes of information with Big Data charactertistics, (Volume, Speed, and Variety).
But nevertheless, until Kylin appeared in the market, OLAP technologies was limited to Relational Databases, or in some cases optimized for multidimensional storage, with serious limitations on Big Data.
Apache Kylin, builded on top of many technologies of Hadoop environment, offer an SQL interface that allows querying data set for multidimensional analysis, achieving response time of a few seconds, over 10 millios rows.
There are keys technologies for Kylin; Apache Hive and Apache HBase
The Data Warehouse is based on a Start Model stored on Apache Hive. 
Using this model and a definition of a meta-data model, Kylin builds a multidimensional MOLAP Cube in HBase. 
After the cube is builded the users can query it, using an SQL based language with its JDBC driver.
When Kylin receives an SQL query, decide if it can be resolved using the MOLAP cube in HBase (in milliseconds), or not, in this case Kylin build its own query and execute it in the Apache Hive Storage, this case is rarely used.
As Kylin has a JDBC driver, we can connect it, to most popular BI tools, like Tableau, or any framework that uses JDBC.

PowerBI:



Power BI is a set of Business Intelligence (BI) tools created by Microsoft. Due to its simplicity and powerful, this emerging tools are becoming a leader BI technology like others such as Tableau, Pentaho or Microstrategy. 
Like these technologies, Power BI is a self-service BI tool, extremely simple but with a lot of powerful features as the following: dashboard developing (called reports in Power BI), web and intra organization sharing and collaborative work, including dozens of powerful charts (ej. line chart with forecasting on page 2 of our demo), connection to relational and Big Data sources, support for natural language Q & A, support to execute and visualize R statistic programs or data preprocessing (ETL).
The above features are implemented across the different tools of Power BI suite. Power BI desktop is a desktop tool for data discovery, transformation and visualization. It is a completely free tool with connectors to the most used relational and Big Data sources. Although for same data sources there are specific connectors, with Apache Kylin we have to use the ODBC connector available on Apache Kylin web page. In this way, we connect to Kylin and a data extract from data source is automatically generated by Power BI. 
At this moment we can create our demo visualization as follows: i) define data model, ii), apply some data transformations if needed (e.g. date format), iii) generate calculated metrics (e.g. student success rate), and then, iv), create the dashboard visualization, with one or multiple pages (e.g. our demo has two page interchangeable with bottom bar selector).
At this time, we have used Power BI service (cloud) to publish on the web our new dashboard join with data extract. To this end, we created an account of Power BI free. In this case, there are also Pro and Premium commercial editions with additional features like data extraction automatic refreshing and direct connections to some data sources such as SQL Server (also Analysis Services), Oracle or Cloudera Impala. 
However none of these direct connectors are for Apache Kylin, then with Kylin we have to use data extraction and data extract refreshing approaches.  
In addition to Power BI Desktop and Power BI Services (Free, Pro and Premium) there are other Power BI tools such as Power BI Mobile (access to dashboard from smartphone and collaborative work) or Power BI Embedded (to use visualizations in ad-hoc apps, web portals, etc).

If you are interested to implement your BI company project with Power BI do not hesitate to contact us on StrateBI.


Open Source Business Intelligence tips in October 2017

10 oct. 2017

Pentaho 8 Reporting for Java Developers


Gracias a Packt que nos ha enviado una copia de: 'Pentaho 8 Reporting for Java Developers' para revisión, como hemos hecho en otras ocasiones y que publicaremos proximamente

Este libro está escrito por un buen amigo con el que hemos coincidido en bastantes Pentaho Developers, Francesco Corti. Echad un vistazo a su web, gran experto en Alfresco y su integración con Pentaho.

Más de 400 páginas de utilidad en este libro, con código para ejercicios

Puedes ver también, el tutorial gratuito sobre Pentaho

5 oct. 2017

Cuales son las novedades es MySQL 8.0?



MySQL, the popular open-source database that’s a standard element in many web application stacks, has unveiled the first release candidate for version 8.0.
Features to be rolled out in MySQL 8.0 include:
  • First-class support for Unicode 9.0 out of the box.
  • Window functions and recursive SQL syntax, for queries that previously weren’t possible or would have been difficult to write.
  • Expanded support for native JSON data and document-store functionality.
With version 8.0, MySQL is jumping several versions in its numbering (from 5.5), due to 6.0 being nixed and 7.0 being reserved for the clustering version of MySQL.

MySQL 8.0’s expected release date

MySQL hasn’t committed to a release date for MySQL 8.0, by MySQL’s policy is “a new [general] release every 18-24 months.” The last general release was October 21, 2015, for MySQL 5.7, so MySQL 8.0’s production version is likely to come in October 2017

Where to download MySQL 8.0
You can download the beta versions of MySQL 8.0 now for Windows, MacOS, several versions of Linux, FreeBSD, and Solaris; the source code is also available. Scroll down the downloads page and go to the Development Releases tab to get them.

Visto en Infoworld

1 oct. 2017

Google lanza Cloud Dataprep in public beta



Muy interesante esta iniciativa de Google en Cloud, Cloud Dataprep, con la idea de facilitar los procesos ETL. Os dejamos la info más abajo, pero según nuestra opinión, dos temas importantes a considerar:

- Data preparation es un eufemismo para intentar dar a entender que los procesos ETL pueden ser sencillos y para usuarios finales, algo que para cualquiera que se dedique al Analytics sabe que no lo es (de hecho, es la parte más compleja e importante, es como la parte oculta de un iceberg). Y, esto es, por que se vislumbra mercado/ingresos en este área. Ver siguiente punto:

- Tiene un modelo de pricing

Google Cloud Dataprep is an intelligent data service for visually exploring, cleaning, and preparing structured and unstructured data for analysis. Cloud Dataprep is serverless and works at any scale. There is no infrastructure to deploy or manage. Easy data preparation with clicks and no code.

The stories behind the data



No dejéis de echar un vistazo a esta iniciativa de Bill Gates: The stories behind the data

"We are launching this report this year and will publish it every year until 2030 because we want to accelerate progress in the fight against poverty by helping to diagnose urgent problems, identify promising solutions, measure and interpret key results, and spread best practices.
As it happens, this report comes out at a time when there is more doubt than usual about the world’s commitment to development."

Microsoft lanza nuevas herramientas de Machine Learning



Microsoft, just like many of its competitors, has gone all in on machine learning. That emphasis is on full display at the company’s Ignite conference this where, where the company today announced a number of new tools for developers who want to build new A.I. models and users who simply want to make use of these pre-existing models — either from their own teams or from Microsoft.

For developers, the company launched three major new tools today: the Azure Machine Learning Experimentation service, the Azure Machine Learning Workbench and the Azure Machine Learning Model Management service.


In addition, Microsoft also launched a new set of tools for developers who want to use its Visual Studio Code IDE for building models with CNTK, TensorFlow, Theano, Keras and Caffe2. And for non-developers, Microsoft is also bringing Azure-based machine learning models to Excel users, who will now be able to call up the AI functions that their company’s data scientists have created right from their spreadsheets.

Visto en Techcrunch

Te puede interesar: Las 53 claves para conocer Machine Learning