Monday, December 26, 2011

BIP Quartz tables

On a 11g BIP only install you can use the QUARTZ scheduler. The scheduler is fed from the XMLP_SCHED_JOB table from the repository. It will seed either the QRTZ_SIMPLE_TRIGGERS (one off jobs) or QRTZ_CRON_TRIGGERS (repeating jobs). This will feed the QRTZ_TRIGGERS table which controls the next execution time. The actual job is in the QRTZ_JOB_DETAILS table.

image

You can inject jobs directly into these tables. Remember that all DateTimes are in QUARTZ_TICKS calculated back to GMT time. A ms-sql conversion function DateTime to Ticks can be foud here. From Ticks to DateTime is this one.

Till Next Time

Friday, December 23, 2011

BIP11g Users and Roles

Did some work on a 11g BI publisher only install recently. Opposite to OBIEE11g you can still add users directly into BIP. Be sure to give them the right roles and sync it with your weblogic security realm.

Till Next Time

Monday, December 19, 2011

OBIEE11g / BIP Quartz Scheduler

Seems that Oracle is using there own standard for the Date Time format in the BIP quartz scheduler. They are not using standard ticks (starting on 1-jan-0001) but a milliseconds count starting on 1-jan-1970. So 1-Jan-2012 = 1325376000000. IE: a whole day is 86400000 ticks.

Till Next Time

Friday, December 16, 2011

OBIEE10g Auto Suggest Prompt

A client asked me if I could create an auto suggest prompt for him. (ie: Google Style Prompt). Basically he wanted an edit box prompt which would fill an suggestion box which he could tab trough to make the right selection.

Since this isn’t a standard 10g functionality I wrote some JavaScript to make it happen. But I didn’t reinvent the wheel Knipogende emoticon ! The people at jQuery already did the bases, I simple adapted it for usage in OBIEE 10g.

1. Download the jQuery UI package here. Install it in your b_mozilla directory’s (or other webserver dirs you use).

2. Download the jQuerySetup from here.

3. Add the setup script to a textbox on your dashboard page:

image

Alter files locations if needed, don’t forget the Contains HTML Markup checkbox.

4. Add a dropdown prompt to your dashboard page.

image

5. Create a javascript file in your b_mozilla directory’s called: autocomplete.js

function SetAutoComplete(PromptColumn){
    var domNode = document;
    var tagName = '*';
    var tags = domNode.getElementsByTagName(tagName);
    var y ="";           
    for(i=0; i<tags.length; i++){
   
    if (tags[i].className  == 'GFPFilter') {
        if (tags[i].getAttribute('gfpbuilder').indexOf(PromptColumn) != -1)
        {   
            y = tags[i].getAttribute('sid')           
        };

        $(
        function()
        {
                $( "#"+y ).combobox();           
        });
        };   
    };
};

6. After the dropdown prompt add a textbox with:

<script src="res/b_mozilla/autocomplete.js" language="javascript"> </script>
<script language="javascript"> 
    SetAutoComplete('C1  Cust Name');
</script>

7. Add your report and run the dashboard:

image

Till Next Time

Tuesday, December 13, 2011

OBIEE10g AutoRunPrompt

A client asked me if I could create an auto run prompt for him. Basically he wanted an edit box prompt which would updated his report after each character has been typed. Since this isn’t a standard 10g functionality I wrote some JavaScript to make it happen. It uses the onkeyup event to fire the GFPDoFilters filter event.

The script can be downloaded here: download COBIEEJS.

Copy the file to your b_mozilla directory’s (or other webserver dirs you use)

How to use it?

1. Add an edit box style prompt to your dashboard:

image

2. Add a textbox with:

<script src="res/b_mozilla/cobieejs.js" language="javascript"> </script>
<script language="javascript"> 
  AutoRunPrompt('C1  Cust Name');
</script>

image

don’t forget the Contains HTML Markup checkbox

3. Add your prompted report:

image

Run the dashboard:

image

Till Next Time

Monday, October 24, 2011

OBIEE11g SampleApp OID doesn’t start

Yep, I did again :-(.

After playing around with the new sampleapp107 I managed to crash the VM. After a reboot the OID refused to start up. It seemed that because there was still an active status record in the ODS schema, the OPMN couldn’t find anything to start.

Solution: truncate the "ODS"."ODS_PROCESS_STATUS" and  "ODS"."ODS_GUARDIAN" tables in your database repository.

(Not sure if the "ODS"."ODS_GUARDIAN"  is really necessary…. Please correct me if I’m wrong)

Till Next Time

Friday, October 21, 2011

OBIEE11g Aggregate At

One of the new feature of 11g is the AGGREGATE AT function. It uses the hierarchical level to pin the the aggregate. Problem is you can’t select a hierarchy level in the formula editor, so you have to some old school typing:

image 

Example:

image

Only month 1 to 6 are selected.

Just like the BY statement you can do multiple levels from different Hierarchies:

image

Till Next Time

Tuesday, October 18, 2011

OBIEE11g Golden Rules: Catalog Management

First of al the original inspiration for these “Golden Rules” Series are based on the “20 GOLDEN RULES FOR REPOSITORY DESIGN” from the people at Peak Indicators. Kudos to them.

The series contains:

The “rules” is this article are somewhat in random order

This is always a “work in progress” and please feel free to make any suggestions!

Catalog management

- use a transport folder.

imageDon’t allow all developers to place everything in all shared folders. Have them place it first in “transport” folder. Assign a “librarian” for the shared folder who will check everything and place it in the correct shared folder.

- Add Metadata to the folders:

image

This makes them better searchable.

- Add structure to the portal:

There are a number of different ways that a portal could be structure to improve efficiency to business users. For instance:

  • Reports could be stored by subject area, such as 'finance', 'sales', 'supply chain'
  • Reports could be stored in a 'daily', 'weekly', 'monthly' directory structure depending on how often they have been designed to be refreshed.
  • Reports could be stored by perspective:
    • Financial — Groups objectives, initiatives, and KPIs that relate to or support the monetary or economic health and development of your organization.
    • Customer — Groups objectives, initiatives, and KPIs that pertain to or support your client base.
    • Internal Process — Groups objectives, initiatives, and KPIs that categorize and support your corporate internal policies and procedures.
    • Learning and Growth — Groups objectives, initiatives, and KPIs that relate to or support employee training and advancement.
  • A combination of the above.

Till Next Time

Saturday, October 15, 2011

OBIEE11g Golden Rules: Dashboard Building

First of al the original inspiration for these “Golden Rules” Series are based on the “20 GOLDEN RULES FOR REPOSITORY DESIGN” from the people at Peak Indicators. Kudos to them. I just added my own observations.

The series contains:

The “rules” is this article are somewhat in random order

This is really a work in progress, will be updated soon! (Bit pressed for Time Knipogende emoticon  )

Dashboard Design

- Don’t crowd the dashboard

Divide your report over several pages! Check why a user really needs 25 pages…..

- Avoid scrolling

Remember that your developers screen is often bigger then the user screen

- Check for screen size

If the user has only a 1024 * 768, then make your own resolution the same!

- Check for mobile

- Animation

- Colours

Before you start using non standard colours have a look a some colour theory: http://en.wikipedia.org/wiki/Color, not every colour combination does well…..

This is always a “work in progress” and please feel free to make any suggestions!

Till Next Time!

Wednesday, October 12, 2011

OBIEE11g Golden Rules: Report Building

First of al the original inspiration for these “Golden Rules” Series are based on the “20 GOLDEN RULES FOR REPOSITORY DESIGN” from the people at Peak Indicators. Kudos to them.

The series contains:

The “rules” is this article are somewhat in random order.

- First column: TIME/CALENDAR

The first column you select for your report should always a column from your time or calendar dimension. Time is often the most consistent aggregation splitter. And most database use partitioning along a time-line.

image

- Dimension Order:

Try to maintain the same order of the dimension across all reports build on a single presentation layer. That way the use of aggregate tables and or query rewrite can be optimally provisioned.

- Move Complex Logic to the rpd:

If you have made a nice “fancy” formula which you probably need to use in a couple of reports, consider moving it to the rpd. That way you only have to maintain it in one place.

- Less is more 1!

If the user is only interested in the “bad” records, then only show hem/her the “bad” records. To make the user scroll trough hundreds of records looking for the ones you flagged with conditional formatting isn’t very efficient.

imageimage

- Less is more 2!

If a report generates more then a hundred records, changes are big that the user is going to do “download to excel”. Check with the user if he needs the report in this form. Consider using different deliver methods (agents / Bip).

- Less is more 3!

If on opening the report the user already has to scroll or navigate to other pages try opening the report on a “higher”  level.

- Avoid multidimensionality on graphs:

If the human eye and brain need to pick up more then 1 dimension on graph it’s easily fooled.

image

- Check graph for “lost” data

image

2011 Paid amount is not visible…..

- Make sure the description is always entered:

A good description must be readable for a “noob”.See: http://obiee101.blogspot.com/2011/09/obiee-mandatory-description-field.html

- Give the report a sell by date!

Go back to the user every 6 to 9 months to see if the report still is required in it’s current version.

This is always a “work in progress” and please feel free to make any suggestions!

Till Next Time

Sunday, October 9, 2011

OBIEE11g Golden Rules: RPD-Presentation Layer

First of al the original inspiration for these “Golden Rules” Series are based on the “20 GOLDEN RULES FOR REPOSITORY DESIGN” from the people at Peak Indicators. Kudos to them.

The series contains:

The “rules” is this article are somewhat in random order

This is always a “work in progress” and please feel free to make any suggestions!

Presentation Layer

- Common dimension

When you have multiple Subject Areas, list the common dimensions in the same order  across all the Subject Areas

imageimage

- Time dimension first:

Since the time/calendar dimension is often the main aggregator make it the first in your presentation layer list.

- No prefixes:

Presentation Table names within each Subject Area must not begin with “Dim – “ or “Fact –“ or “Fact Compound –“. So remove these prefixes if they are present after creating the Subject Area by dragging Logical Tables directly from the Business Model.

- Identify your facts:

The Presentation Table containing your facts should be listed right at the bottom, and the Presentation Table name should contain words like “Measures” or “Facts”

image

- Ensure logical relationship:

There should be absolutely no possibility whatsoever of a user selecting objects from a Subject Area that have no logical relationship. So, if there are any objects within the same Subject Area that cannot co-exist in the same report, then your Subject Area design is incorrect!

- Split over multiple subject area:

Within OBIEE11g report can be build using multiple presentation layers based on the same business layer:

image Consider splitting your presentation layer in “sub” areas.

- Dimension Column Order:

Try to have the column in the same order as your hierarchy: Year > Quarter > Month > Week > Date or Business Line > Brand > Product

- Special characters:

Special HTML characters {< > / } should be avoided in the object names. Not ever browser can render them correctly.

- Metadata dictionary

Have a well maintained metadata dictionary in place:

image

Remember in OBIEE11G you have to redeploy the metadata dictionary after each RPD deployment

Till Next Time

Thursday, October 6, 2011

OBIEE11g Golden Rules: RPD-Business Model Layer

First of al the original inspiration for these “Golden Rules” Series are based on the “20 GOLDEN RULES FOR REPOSITORY DESIGN” from the people at Peak Indicators. Kudos to them.

The series contains:

The “rules” is this article are somewhat in random order

This is always a “work in progress” and please feel free to make any suggestions!

Business Model Layer

- Prefix Logical Tables

All Logical Tables should be prefixed. There are several naming convention's in use:

  • “Dim – “, “Fact – “ or “Fact Compound –“
  • “D## name”, “F## name” or “FC## name”

image

- No “physical” column names

No “physical” column names should ever be seen on the Business Model layer. All naming conventions should be “business oriented”. For example use “$ Revenue” rather than “DOLLARS” .

- No Primary or Surrogate Physical Keys

Physical Primary Keys or Surrogate Keys should not be present on the Business Model layer (unless, for example, you have a Primary Key such as Order Id which will be displayed on reports)

- Logical Keys

Dimension Logical Tables must always have a Logical Key assigned. The Logical Key should be something “business oriented” such as “Employee Login” rather than “EMPLOYEE_PK”

image

- No Facts in dimensions

Dimension Logical Tables must only contain dimension attributes, they should never contain any measure columns (which have an Aggregate Rule)

- No Logical Keys on facts

Fact Logical Tables should not have a Logical Key assigned.

image
[UPDATE: ] As far as I know this is because the OBIEE optimizer uses the logical key to determine the "driving" table. No logical keys on fact tables should ensure the "correct" optimizer path. Please correct me if I’m misinformed.

- Aggregation Rules on Facts

Every Logical Column within a Fact Logical Table must be a measure column, and therefore have an Aggregation Rule assigned.

image

- Only Complex Joins

When defining Logical Joins between Logical Tables, only use “Complex Joins” (and use the default settings – you only ever specify a “Driving Table” when dealing with cross-database joins)

- No Snowflakes

The Business Model should only consist of logical star-schemas, there should not be any snow-flaking

image

- Hierarchies on Dimensions

Every Dimension Logical Table should have a corresponding Dimension Hierarchy (with “Total” as a Grand Total level, and “Detail” at the lowest level)

image

- Number of Elements

Each level of a Dimension Hierarchy should have its “Number of Elements” appropriately set (there is a utility in Tools that can do this automatically).

 image

- Content Levels

Every Logical Table Source within every dimension and fact Logical Table should have its “Content Levels” appropriately set. The only time the “Content Level” is not set for a particular dimension is when there is no logical relationship existing

image

- Multiple facts tables

Do not merge all your measures into a single Fact Logical Table. For example, you should split “Forecast Sales” and “Actual Sales” measures into two Logical Tables e.g. “Fact – Sales” and “Fact – Forecast”

image

- Description fields

All available description field should have meaningful descriptions with non technical users.

Till Next Time

Monday, October 3, 2011

OBIEE11g Golden Rules: RPD-Physical Layer

First of al the original inspiration for these “Golden Rules” Series are based on the “20 GOLDEN RULES FOR REPOSITORY DESIGN” from the people at Peak Indicators. Kudos to them.

The series contains:

  • RPD-Physical Layer
  • RPD-Business Model Layer
  • RPD-Presentation Layer
  • Catalog Structure
  • Report Building
  • Dashboard Building

The “rules” is this article are somewhat in random order

This is always a “work in progress” and please feel free to make any suggestions!

Physical Layer

- Clear the cache check box

image

Using cache should be a last resource, reconsider your data model and ETL processes first!

 

 

 

- Always use “Foreign Key” joins, not “Complex Joins” on the Physical Layer

If your join looks like D_DATE = TRUNC(S_DATETIME) try add a extra column S_DATE in your DWH. Any matching processing done by the BI-server costs time and you often loose the advantage of an index in your DWH.

- Prefix your tables

When modelling a star-schema data-model, create aliases for all your physical tables (prefixed with either “Dim_”, “Fact_” or “Fact_Agg_” )

image

- Physical Display Folder

Use Physical Display Folder to organise your stars

image

- Call Interface

When possible, configure your connection pools to use a “native driver” to connect to your physical databases. For example, use OCI for connecting to an Oracle database rather than ODBC.
image

- Parameterize your data source name

image

This way you only have to change it in one place when moving from development to production.

- 3NF in DWH

Try to avoid doing 3NF to Star Schema Modelling in the Physical Layer. Flatten the table if possible during the ETL or in a view on the database.

- Connecting User

image

The User you use to connect to your data should by default not by the “owner” of the table, but should have only select rights trough a role.

Till Next Time

Saturday, October 1, 2011

OBIEE obips_config_base.xsd

Good Sunday afternoon reading, locate you obips_config_base.xsd and open it read only in a text editor (or download the PDF here).

It' contains loads of info on the instanceconfig.xml tags.

Have a special look at the  <!-- Top Secret, Do NOT Expose On Penalty Of Dismissal (throw away) –> stuff

image 

(Kudos to the genius who left them in the normal download Knipogende emoticon

Till Next Time

Friday, September 30, 2011

OBIEE Portal management

First of all the main source for this article can be found here: http://www.datafactotum.com/2010/01/how-to-ensure-efficient-use-of.html a must read article from Phil Wright .

In this article I want to give you some guidelines for the implementation in OBIEE. (It’s still a work in progress so feel free to add any comments or suggestions)

The OBIEE portal should be considered essential component of the BI Landscape of an organisation. It can be a great delivery method to allow the business to serve itself with accurate, timely, insightful information, without the need for further IT intervention, delays, or confusion. (hopefully Knipogende emoticon )

Looking around in the organisations I worked with I see that id they are not properly managed they tend to become a “garden shed” after a while:

image

A place where only a few people can find what they need….

 

 

 

 

 

Within a couple of months you will get a “buzz”  like:

  • The information seems to be out of date…
  • Is this the same report as this one?
  • This report still says “test” but department X is already using it…
  • What is the purpose of this report
  • Who owns this report?
  • This report is missing information on X, Y & Z
  • I thought we changed the procedure for this……

If the “buzz” grows to much it will lead to a lack of confidence among business users in the BI system.

How can we, as OBIEE specialists ensure that our portal breeds confidence and remains an efficient delivery method of information to business?

Based on Phil’s original points I come to the following list: (please feel free to add!)

  • Set up a commination model
  • Organize your Catalog
  • Template your dashboards
  • Monitor 'last refreshed' or 'last accessed' date
  • Introduce increased governance
  • Introduce Kite marking
  • Allow search of reports by keywords
  • Ensure accurate metadata is in place
  • Introduce structure to portal

Set up a commination model

Users tend to forget how things work especially procedure’s Knipogende emoticon.

image

Set up a wiki / sharepoint were you put a related BI documentation. Add a hyperlink on each dashboard page.

Send out a regular newsletter on your progress and all the exiting new things you have made

Organise your catalog:

image

  • Use catalog groups: Control access to your shared folders
  • Create Functional Folder: Group reports and dashboards
  • Create a Transport Folder: Reports from individual users which have to become cooperate can be placed here. The portal manager will put them in the correct destination folder.
  • A you sure everybody needs to see every page?

Template your dashboards:

Make sure your dashboards ‘look and feel’ the same throughout the whole portal.

  • Start with a landing page
  • Include a help/explanation page
  • Include last refresh date of the DWH.
  • Restrict the number of used graph type’s
  • don’t cramp to much into one page
  • don’t go overboard with animations
  • avoid scrolling

Monitor 'last accessed' date

The usage tracking feature in OBIEE gives you the ability to look at dates that reports where last accessed. Temporarily remove all reports/dashboard that have not been accessed within the past 3 to 6 months. These reports need to be fenced off into a temporary storage area for a couple of months, and if a business user does not complain that their report is missing, the report probably can permanently deleted (read archived). Due to the ever changing nature of business, reports are often created for a specific purpose, used heavily for a period of time, and then no longer deemed necessary, or are superseded by a new report. If this ‘old’ content remains available for each user on the portal, it will start confusing users.

Introduce increased governance

The BI portal project will of course have started with the best intentions in terms of governance, with the initial batch of published reports having defined owners, purpose, scheduling, definition etc.

However, when the portal grows, this governance benchmark often slips away if there is no adequate management in place.

A regular exercise of updating and re-introducing the governance standards to the portal, as well as providing education for the standards of future reports that will be published will help ease business concerns.

Don’t forget to communicate and embrace the standard with the business on a regular basis.

A tip is to introduce the ‘report of the month’ , send a mail to the organisation telling them which spectacular new page has been added.

The minimum documentation on reports/dashboards from the business perspective should be based on the 4 streams mentioned in Phil article:

image

  • Purpose
    - Identify why report is needed (operational, KPI, KQI)
    - Identify what the business need is
    - Identify how it will be used
  • Ownership
    - Who will own the report?
    - Who is the primary contact?
    - Are they also responsible for potential report issues?
  • Classification
    -
    Is the report sensitive?
    - Does it require special permissions to view?
    - Is a confidentiality agreement needed?

  • Definition
    -
    What do the fields on the report mean?
    - Who has aided in defining the fields?
    - Has definition been signed-off as correct?

These 4 streams form the basis of a 'Standards in Report Creation & Publishing' document on your company wiki.

Introduce Kite/CE/TQCSI marking

imageimageimage

A further step into the world of report governance could be taken in the form of quality marking reports.

Marking is the process of stamping a report as a recognised source of accurate and approved data. This way the business users would know that the report contains information of which they can be confident will support their business decisions.

Included a ‘VALID TILL’/’NEXT REVIEW’/’OVERHAUL’ date  on the report. image

Reports seldom should live forever!

Make content searchable

OBIEE has a great search feature, allowing user to find the report they need:

image ….But you need to ensure that metadata exists that allows a business user to easily understand, in business terms, what the report shows, and how it should be used.

A good tip to let your user write (part of) the documentation.

This means:

Ensure accurate metadata is in place

Metadata has different meaning for different users. Beside the ‘technical’ side you should also specify metadata from a business perspective, such as:

image

  • - A meaningful name of the report
  • - Defined business terms for each field of a report
  • - Information related to report owner
  • - Business rules and any criteria applied to the report are clearly defined

These forms of metadata, where applicable, should be captured either on the report, or within a separate business glossary / wiki page.

Due to the ever-changing nature of business, ensuring accurate metadata can be harder than it seems. Over time definitions and business rules can change. It is therefore essential that a regular routine exercise of maintaining metadata is undertaken. A report with out of date definitions or business rules could lead to data quality issues and poor decisions.

Set up a metadata review every 6 till 9 months.

Introduce structure to portal

Structure is essential to providing an efficient portal to business users. There is nothing worse than having to scroll through a list of 400 reports to find the report you're looking for.

There are a number of different ways that a portal could be structure to improve efficiency to business users. For instance:

  • Reports could be stored by subject area, such as 'finance', 'sales', 'supply chain'
  • Reports could be stored in a 'daily', 'weekly', 'monthly' directory structure depending on how often they have been designed to be refreshed.
  • Reports could be stored by perspective:
    • Financial — Groups objectives, initiatives, and KPIs that relate to or support the monetary or economic health and development of your organization.
    • Customer — Groups objectives, initiatives, and KPIs that pertain to or support your client base.
    • Internal Process — Groups objectives, initiatives, and KPIs that categorize and support your corporate internal policies and procedures.
    • Learning and Growth — Groups objectives, initiatives, and KPIs that relate to or support employee training and advancement.
  • A combination of the above.

Think of the portal as a bookshelf, or a library, with the aim of enabling the business user to find the correct information they require in a timely manner. This implies that different users may need different portal organisation structures.

And the big question is: …..

Although most of these points seem to be based on common sense, the big question is often who should implement and maintain these guidelines within an organisation?

image

I personally think this is one of the core a activities of a BICC (Business Intelligence Competence Center) see: http://en.wikipedia.org/wiki/Business_Intelligence_Competency_Center

 

Till Next Time