Material Big Data

Lanzados ppts informativos de tecnologías BigData: Hadoop, Hbase, Hive, Zookeeper...

Se convoca Curso Business Intelligence Open Source. No te lo pierdas!!

Quieres aprender sobre Pentaho, Mondrian, Saiku, CTools, Talend y más... de los expertos?

Pentaho Analytics. Un gran salto

Ya se ha lanzado Pentaho 6 y con grandes sorpresas. Descubre con nosotros las mejoras de la mejor suite Open BI

La mejor oferta de Cusos Open Source

Después de la gran acogida de nuestros Cursos Open Source, eminentemente prácticos, lanzamos las convocatorias de 2016

25 de abril de 2016

Ranking de principales usos y tecnologias del Big Data


Se habla mucho de Big Data, sus tecnologías, aplicaciones, etc... pero resulta difícil concretar el principal uso que le podemos dar, así como su nivel de madurez. 

A continuación se indican las principales tecnologías en las que tiene una aplicación real:
  1. Predictive analytics: software and/or hardware solutions that allow firms to discover, evaluate, optimize, and deploy predictive models by analyzing big data sources to improve business performance or mitigate risk.
  2. NoSQL databases: key-value, document, and graph databases.
  3. Search and knowledge discovery: tools and technologies to support self-service extraction of information and new insights from large repositories of unstructured and structured data that resides in multiple sources such as file systems, databases, streams, APIs, and other platforms and applications.
  4. Stream analytics: software that can filter, aggregate, enrich, and analyze a high throughput of data from multiple disparate live data sources and in any data format.
  5. In-memory data fabric: provides low-latency access and processing of large quantities of data by distributing data across the dynamic random access memory (DRAM), Flash, or SSD of a distributed computer system.
  6. Distributed file stores: a computer network where data is stored on more than one node, often in a replicated fashion, for redundancy and performance.
  7. Data virtualization: a technology that delivers information from various data sources, including big data sources such as Hadoop and distributed data stores in real-time and near-real time.
  8. Data integration: tools for data orchestration across solutions such as Amazon Elastic MapReduce (EMR), Apache Hive, Apache Pig, Apache Spark, MapReduce, Couchbase, Hadoop, and MongoDB.
  9. Data preparation: software that eases the burden of sourcing, shaping, cleansing, and sharing diverse and messy data sets to accelerate data’s usefulness for analytics.
  10. Data quality: products that conduct data cleansing and enrichment on large, high-velocity data sets, using parallel operations on distributed data stores and databases.

Visto en Forbes

22 de abril de 2016

Listado completo de Open Data Portals en el mundo



Open Data Inception es una web imprescindible para todos los que trabajamos con Datos Abiertos. Muy recomendable

17 de abril de 2016

Ranking de las mejores Bases de Datos

Mas de 300 bases de datos son evaluadas en la comparativa que realizan en DB-engines anualmente

Un imprescindible para todos los que manejan datos. Cada vez tenemos más opciones y tecnologías donde elegir. Ah, y la mayoría, son Open Source



14 de abril de 2016

Preguntas clave en una entrevista para conocer si sabes Pentaho

Muchos os preguntaréis, cuales son los principales conocimientos que se pueden evaluar en un especialista para trabajar con Pentaho, aquí os damos unas pistas:

1.Define Pentaho and its usage.

Revered as one of the most efficient and resourceful data integration tools (DI), Pentaho virtually supports all available data sources and allows scalable data clustering and data mining. It is a light-weight Business Intelligence suite executing Online Analytical Processing (OLAP) services, ETL functions, reports and dashboards creation and other data-analysis and visualization operations.

2.Explain the important features of Pentaho.

• Pentaho is capable of creating Advanced Reporting Algorithms regardless of their input and output data format.
• It supports various report formats, whether Excel spreadsheets, XMLs, PDF docs, CSV files.
• It is a Professionally Certified DI Software rendered by the renowned Pentaho Company headquartered in Florida, United States.
• Offers enhanced functionality and in-Hadoop functionality
• Allows dynamic drill down into larger and greater information
• Rapid Interactive response optimization
• Explore and view multidimensional data

3.Name major applications comprising Pentaho BI Project.

• Business Intelligence Platform
• Dashboards and Visualizations
• Reporting
• Data Mining
• Data Analysis
• Data Integration and ETL (also called Kettle)
• Data Discovery and Analysis (OLAP)

4.What is the importance of metadata in Pentaho?

A metadata model in Pentaho formulates the physical structure of your database into a logical business model. These mappings are stored in a central repository and allow developers and administrators to build business-logical DB tables that are cost effective and optimized. It further simplifies the working of business users allowing them to create formatted reports and dashboards ensuring security to data access.
All in all, metadata model provides an encapsulation around the physical definitions of your database and the logical representation and define relationships between them.

5.Define Pentaho Reporting Evaluation.

Pentaho Reporting Evaluation is a particular package of a subset of the Pentaho Reporting capabilities, designed for typical first-phase evaluation activities such as accessing sample data, creating and editing reports, and viewing and interacting with reports.

6.Explain the benefits of Data Integration.

• The biggest benefit is that integrating data improves consistency and reduces conflicting and erratic data from the DB. Integration of data allows users to fetch exactly what they look for, enabling them utilizeand work with what they collected.
• Accurate data extraction, which further facilitates flexible reporting and monitoring of the available volumes of data
• Helps meet deadlines for effective business management
• Track customer’s information and buying behavior to improve traffic and conversions in the future, thus advancing your business performance.

7.What is MDX and its usage?

MDX is an acronym for ‘Multi-Dimensional Expressions,’ the standard query language introduced by Microsoft SQL OLAP Services. MDX is an imperative part of XML for analysis API, which has a different structure than SQL. A basic MDX query is:
SELECT {[Quantity].[Unit Sales], [Quantity].[Store Sales]} ON COLUMNS,
{[Product].members} ON ROWS
FROM [Sales]
WHERE [Time].[1999].[Q2]

8.Define three major types of Data Integration Jobs.

• Transformation Jobs: Used for preparing data and used only when the there is no change in data until transforming of data job is finished
• Provisioning Jobs: Used for transmission/transfer of large volumes of data. Used only when no change is data is allowed unless job transformation and on large provisioning requirement.
• Hybrid Jobs: Execute both transformation and provisioning jobs. No limitations for data changes; it can be updates regardless of success/failure. The transforming and provisioning requirements are not large in this case.

9.Illustrate the difference between transformations and jobs.

While transformations refer to shifting and transforming rows from source system to target system, jobs perform high level operations like implementing transformations, file transfer via FTP, sending mails, etc.
Another significant difference is that the transformation allows parallel execution whereas jobs implement steps in order.

10.How to perform database join with PDI (Pentaho Data Integration)?

PDI supports joining of two tables form the same databse using a ‘Table Input’ method, performing the join in SQL only.
On the other hand, for joining two tables in different databases, users implement ‘Database Join’ step. However, in database join, each input row query executes on the target system from the main stream, resulting in lower performance as the number of queries implement on the B increases.
To avoid the above situation, there is yet another option to join rows form two different Table Input steps. You can use ‘Merge Join ‘step, using the SQL query having ‘ORDER BY’ clause. Remember, the rows must be perfectly sorted before implementing merge join.

11.Explain how to sequentialize transformations?

Since PDI transformations support parallel execution of all the steps/operations, it is impossible to sequentialize transformations in Pentaho. Moreover, to make this happen, users need to change the core architecture, which will actually result in slow processing.

12.Explain Pentaho Reporting Evaluation.

Pentaho Reporting evaluation is a complete package of its reporting abilities, activities and tools, specifically designed for first-phase evaluation like accessing the sample, generating and updating reports, viewing them and performing various interactions. This evaluation consists of Pentaho platform components, Report Designer and ad hoc interface for reporting used for local installation. 

13.Can fieldnames in a row duplicated in Pentaho?

No, Pentaho doesn’t allow field duplication.

14.Does transformation allow filed duplication?

“Select Values” will rename a field as you select the original field also.  The original field will have a duplicate name of the other field now.

15.How to use database connections from repository?

You can either create a new transformation/job or close and reopen the ones already loaded in Spoon.

16.Explain in brief the concept Pentaho Dashboard.

Dashboards are the collection of various information objects on single page including diagrams, tables and textual information. The Pentaho AJAX API is used to extract BI information while Pentaho Solution Repository contains the content definitions. The steps involved in Dashboard creation include
• Adding dashboard to the solution
• Defining dashboard content
• Implementing filters
• Editing dashboards

17.How to use logic from one transformation/job in other process?

Transformation logic can be shared using subtransformations, which provides seamless loading and transformation of variables enhancing efficiency and productivity of the system. Subtransformations can be called and reconfigured when required.

18.Explain the use of Pentaho reporting.

Pentaho reporting enables businesses to create structured and informative reports to easily access, format and deliver meaningful and important information to clients and customers. They also help business users to analyze and track consumer’s behavior for the specific time and functionality, thereby directing them towards the right success path.

19.What is Pentaho Data Mining?

Pentaho Data Mining refers to the Weka Project, which consists of a detailed tool set for machine learning and data mining. Weka is open source software for extracting large sers of information about users, clients and businesses. It is built on Java programming.

20.Is Data Integration and ETL Programming same?

No. Data Integration refers to passing of data from one type of systems to other within the same application. On the contrary, ETL is used to extract and access data from different sources. And transform it into other objects and tables.

21.Explain Hierarchy Flattening.

It is just the construction of parent child relationships in a database. Hierarchy Flattening uses both horizontal and vertical formats, which enables easy and trouble-free identification of sub elements. It further allows users to understand and read the main hierarchy of BI and includes Parent column, Child Column, Parent attributes and Child attributes.

22.Give a brief about PDI architecture.

PDI architecture

23.Explain Pentaho report Designer (PRD).

PRD is a graphic tool to execute report-editing functions and create simple and advanced reports and help users export them in PDF, Excel, HTML and CSV files. PRD consists of Java-based report engine offering data integration, portability and scalability. Thus, it can be embedded in Java web applications and also other application servers like Pentaho BAserver.

24.Define Pentaho Report types.

There are several categories of Pentaho reports:
• Transactional Reports: Data to be used form transactions. Objective is to publish detailed and comprehensive data for day-to-day organization’s activities like purchase orders, sales reporting
• Tactical Reports: data comes from daily or weekly transactional data summary. Objective is to present short-term information for instant decision making like replacing merchandize.
• Strategic Reports: data comes from stable and reliable sources to create long-term business information reports like season sales analysis.
• Helper Reports: data comes from various resources and includes images, videos to present a variety of activities.

25.What are variables and arguments in transformations?

Transformations dialog box consists of two different tables: one of arguments and the other of variables. While arguments refer to command line specified during batch processing, PDI variables refer to objects that are set in a previous transformation/job in the OS.

26.How to configure JNDI for Pentaho DI Server?

Pentaho offers JNDI connection configuration for local DI to avoid continuous running of application server during the development and testing of transformations.  Edit the properties in jdbc.propertiesfile located at…\data-integration-server\pentaho-solutions\system\simple-jndi.





Visto en Intellipat

13 de abril de 2016

Ya esta disponible Pentaho 6.1



Ya os podéis descargar la nueva version Pentaho 6.1

Primeras informaciones con las novedades:

Jens Bluel

Matt Casters

Pedro Alves

Pentaho


Seguiremos informando...

12 de abril de 2016

Ejemplos de Business Intelligence Open Source


Hace unos días se celebró el interesante evento organizado por BI-Spain sobre Business Intelligence y visualización

Una de las presentaciones más interesantes fue la de nuestros compañeros de Stratebi que hablaron de como usar Pentaho y otras soluciones Open Source para crear grandes aplicaciones Business Intelligence y de visualización Big Data, real time, Internet of Things, Social Media....

Aquí podéis ver el video de dicha prsesentación en la que se mostraron muy buenos ejemplos



En el siguiente enlace podéis acceder y trabajar directamente con la aplicación


7 de abril de 2016

20 razones que nos llevan a tomar malas decisiones

Buena infografía. Imprimidla y colgarla en la pared


4 de abril de 2016

Comparativa economica y deportiva del Real Madrid y FC Barcelona







Cada vez es más importante para los clubes deportivos conocer el coste económico y rentabilidad de sus grandes jugadores, en relación a su rendimiento deportivo. Pero también conocer su impacto en las redes sociales

Más aún, si comparamos entre eternos rivales como el Real Madrid y el FC Barcelona

En esta web puedes comprobar cuanto le cuesta a cada equipo los goles de Messi y Ronaldo, que delanteras son más rentables, cuanto gana Ramos y Piqué cada hora...



2 de abril de 2016

Comparativa Kettle (Pentaho Data Integration) y Talend

Hace unos días os hablábamos de que el ETL es crucial y hoy os mostramos una comparativa de las dos mejores herramientas Open Source de ETL (Kettle de Pentaho y Talend), que tampoco empieza a ser arriesgado a decir que se están convirtiendo en las mejores, sobre todo si valoramos el coste y la posibilidad de integración y modificación respecto a Informatica Powercenter, Oracle, Microsoft o IBM

Tanto Kettle como Talend son grandes herramientas, muy visuales, que nos permiten integrar todo tipo de fuentes, incluyendo también Big Data para hacer todo tipo de transformaciones y proyectos de integración o para preparar potentes entornos analíticos, también con soluciones Open Source como podéis ver en esta Demo Online, donde se han usado Kettle y Talend en el backend




Descargar la comparativa de Excella 

Business Intelligence Open Source Tips in March




Now you can check latest tips on Business Intelligence Open Source, mainly Pentaho, Ctools and Saiku in March. You can see some of this tips implemented in Demo Online

This month with great stuff:

-          Xavier OLAP client with Jedox