Add extra functionalities

Add LEFT JOIN capabilities

Add sums and avg capabilities for tree views

Robustness and code review

Provide ER diagram view for table relations

bi_view_editor/README.rst

@@ -14,13 +14,13 @@ BI View Editor
     :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
     :alt: License: AGPL-3
 .. |badge3| image:: https://img.shields.io/badge/github-OCA%2Freporting--engine-lightgray.png?logo=github
-    :target: https://github.com/OCA/reporting-engine/tree/11.0/bi_view_editor
+    :target: https://github.com/OCA/reporting-engine/tree/12.0/bi_view_editor
     :alt: OCA/reporting-engine
 .. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
-    :target: https://translation.odoo-community.org/projects/reporting-engine-11-0/reporting-engine-11-0-bi_view_editor
+    :target: https://translation.odoo-community.org/projects/reporting-engine-12-0/reporting-engine-12-0-bi_view_editor
     :alt: Translate me on Weblate
 .. |badge5| image:: https://img.shields.io/badge/runbot-Try%20me-875A7B.png
-    :target: https://runbot.odoo-community.org/runbot/143/11.0
+    :target: https://runbot.odoo-community.org/runbot/143/12.0
     :alt: Try me on Runbot
 
 |badge1| |badge2| |badge3| |badge4| |badge5| 
@@ -45,33 +45,65 @@ Purpose:
 .. contents::
    :local:
 
+Installation
+============
+
+In the Odoo configuration file add ``bi_view_editor`` in the list
+``server_wide_modules``:
+
+.. code-block:: ini
+
+  [options]
+  (...)
+  server_wide_modules = web,bi_view_editor
+  (...)
+
+Alternatively specify ``--load=bi_view_editor`` when starting Odoo by command line.
+
+Optionally it is possible to enable the view of the ER Diagram. For this you
+need to install `Graphviz`, an open source graph visualization software:
+
+.. code-block:: bash
+
+   ``sudo apt-get install graphviz``
+
 Usage
 =====
 
 To graphically design your analysis data-set:
 
 - From the Dashboards menu, select "Custom BI Views"
-- Browse trough the business objects in the Query tab
+- Browse trough the business objects in the "Query Builder" tab
 - Pick the interesting fields (Drag & Drop)
-- For each selected field, right-click on the Options column and select whether it's a row, column or measure; if you want to remove the field from the list view, unflag the checkbox ´List´ in the Options column
+- For each selected field, right-click on the Options column and select whether
+  it's a row, column or measure; if you want to remove the field from the list
+  view, unflag the checkbox ´List´ in the Options column
 - Save and click "Generate BI View"
 - Click "Open BI View" to view the result
-- If module Dashboard (board) is installed, the standard "Add to My Dashboard" functionality would be available
+
-- Click "Create a menu" to create a new menu item directly linked to your new BI view (this feature is available in developer mode); when the BI view is reset back to draft this menu will be removed, and you will need to re-create the menu entry.
+To access the created BI View with a dedicated menu:
+
+- If module Dashboard (board) is installed, the standard "Add to My Dashboard"
+  functionality would be available
+- Click "Create a menu" to create a new menu item directly linked to your new
+  BI view (this feature is available in developer mode); when the BI view is
+  reset back to draft this menu will be removed, and you will need to re-create
+  the menu entry.
+
+A more advanced UI is also available under the "Details" tab. It provides extra
+possibilities for more advanced users, like to use LEFT JOIN instead of the
+default INNER JOIN.
 
 Known issues / Roadmap
 ======================
 
-* Non-stored fields and many2many fields are not supported
+* Non-stored fields and many2many fields are not supported.
-* Provide graph view for table relations
+* Provide a tutorial (eg. a working example of usage).
-* Extend the capabilities of the tree views (e.g. add sums)
+* Find better ways to extend the *_auto_init()* without override.
-* Provide a tutorial (eg. a working example of usage)
+* Possibly avoid the monkey patches.
-* Implement a more advanced UI, with possibilities to use LEFT JOIN as default instead of INNER JOIN
+* Data the user has no access to (e.g. in a multi company situation) can be
-* Find better ways to extend the *_auto_init()* without override
+  viewed by making a view. Would be nice if models available to select when
-* Possibly avoid the monkey patches
+  creating a view are limited to the ones that have intersecting groups.
-* Data the user has no access to (e.g. in a multi company situation) can be viewed by making a view
-* Store the JSON data structure in ORM
-* Would be nice if models available to select when creating a view are limited to the ones that have intersecting groups (for non technical users)
 
 Bug Tracker
 ===========
@@ -79,7 +111,7 @@ Bug Tracker
 Bugs are tracked on `GitHub Issues <https://github.com/OCA/reporting-engine/issues>`_.
 In case of trouble, please check there if your issue has already been reported.
 If you spotted it first, help us smashing it by providing a detailed and welcomed
-`feedback <https://github.com/OCA/reporting-engine/issues/new?body=module:%20bi_view_editor%0Aversion:%2011.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
+`feedback <https://github.com/OCA/reporting-engine/issues/new?body=module:%20bi_view_editor%0Aversion:%2012.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
 
 Do not contact contributors directly about support or help with technical issues.
 
@@ -106,15 +138,10 @@ Contributors
 Other credits
 ~~~~~~~~~~~~~
 
-Images
-------
-
-* Odoo Community Association: `Icon <https://github.com/OCA/maintainer-tools/blob/master/template/module/static/description/icon.svg>`_.
-
 Funders
 -------
 
-The development of this module has been financially supported by:
+The development of this module for Odoo 11.0 has been financially supported by:
 
 * IDEAL Connaissances SAS https://www.idealconnaissances.com
 
@@ -131,6 +158,6 @@ OCA, or the Odoo Community Association, is a nonprofit organization whose
 mission is to support the collaborative development of Odoo features and
 promote its widespread use.
 
-This module is part of the `OCA/reporting-engine <https://github.com/OCA/reporting-engine/tree/11.0/bi_view_editor>`_ project on GitHub.
+This module is part of the `OCA/reporting-engine <https://github.com/OCA/reporting-engine/tree/12.0/bi_view_editor>`_ project on GitHub.
 
 You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

bi_view_editor/models/bve_view.py

@@ -1,7 +1,9 @@
 # Copyright 2015-2019 Onestein (<https://www.onestein.eu>)
 # License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
 
+import base64
 import json
+import pydot
 from psycopg2.extensions import AsIs
 
 from odoo import _, api, fields, models, tools
@@ -31,7 +33,7 @@ class BveView(models.Model):
         for bve_view in self:
             serialized_data = []
             for line in bve_view.line_ids.sorted(key=lambda r: r.sequence):
-                serialized_data_dict = {
+                serialized_data.append({
                     'sequence': line.sequence,
                     'model_id': line.model_id.id,
                     'id': line.field_id.id,
@@ -45,13 +47,9 @@ class BveView(models.Model):
                     'column': line.column,
                     'measure': line.measure,
                     'list': line.in_list,
-                }
+                    'join_node': line.join_node,
-                if line.join_node:
+                    'relation': line.relation,
-                    serialized_data_dict.update({
+                })
-                        'join_node': line.join_node,
-                        'relation': line.relation,
-                    })
-                serialized_data += [serialized_data_dict]
             bve_view.data = json.dumps(serialized_data)
 
     def _inverse_serialized_data(self):
@@ -76,6 +74,16 @@ class BveView(models.Model):
         'bve.view.line',
         'bve_view_id',
         string='Lines')
+    field_ids = fields.One2many(
+        'bve.view.line',
+        'bve_view_id',
+        domain=['|', ('join_node', '=', -1), ('join_node', '=', False)],
+        string='Fields')
+    relation_ids = fields.One2many(
+        'bve.view.line',
+        'bve_view_id',
+        domain=[('join_node', '!=', -1), ('join_node', '!=', False)],
+        string='Relations')
     action_id = fields.Many2one('ir.actions.act_window', string='Action')
     view_id = fields.Many2one('ir.ui.view', string='View')
     group_ids = fields.Many2many(
@@ -89,7 +97,8 @@ class BveView(models.Model):
         string='Users',
         compute='_compute_users',
         store=True)
-    query = fields.Text()
+    query = fields.Text(compute='_compute_sql_query')
+    er_diagram_image = fields.Binary(compute='_compute_er_diagram_image')
 
     _sql_constraints = [
         ('name_uniq',
@@ -97,49 +106,91 @@ class BveView(models.Model):
          _('Custom BI View names must be unique!')),
     ]
 
-    @api.multi
+    @api.depends('line_ids')
+    def _compute_er_diagram_image(self):
+        for bve_view in self:
+            graph = pydot.Dot(graph_type='graph')
+            table_model_map = {}
+            for line in bve_view.field_ids:
+                if line.table_alias not in table_model_map:
+                    table_alias_node = pydot.Node(
+                        line.model_id.name + ' ' + line.table_alias,
+                        style="filled",
+                        shape='box',
+                        fillcolor="#DDDDDD"
+                    )
+                    table_model_map[line.table_alias] = table_alias_node
+                    graph.add_node(table_model_map[line.table_alias])
+                field_node = pydot.Node(
+                    line.table_alias + '.' + line.field_id.field_description,
+                    label=line.description,
+                    style="filled",
+                    fillcolor="green"
+                )
+                graph.add_node(field_node)
+                graph.add_edge(pydot.Edge(
+                    table_model_map[line.table_alias],
+                    field_node
+                ))
+            for line in bve_view.relation_ids:
+                field_description = line.field_id.field_description
+                table_alias = line.table_alias
+                diamond_node = pydot.Node(
+                    line.ttype + ' ' + table_alias + '.' + field_description,
+                    label=table_alias + '.' + field_description,
+                    style="filled",
+                    shape='diamond',
+                    fillcolor="#D2D2FF"
+                )
+                graph.add_node(diamond_node)
+                graph.add_edge(pydot.Edge(
+                    table_model_map[table_alias],
+                    diamond_node,
+                    labelfontcolor="#D2D2FF",
+                    color="blue"
+                ))
+                graph.add_edge(pydot.Edge(
+                    diamond_node,
+                    table_model_map[line.join_node],
+                    labelfontcolor="black",
+                    color="blue"
+                ))
+            try:
+                png_base64_image = base64.b64encode(graph.create_png())
+                bve_view.er_diagram_image = png_base64_image
+            except:
+                bve_view.er_diagram_image = False
+
     def _create_view_arch(self):
         self.ensure_one()
 
-        def _get_field_def(name, def_type):
+        def _get_field_def(line):
-            return """<field name="{}" type="{}" />""".format(name, def_type)
+            field_type = line.view_field_type
+            return '<field name="%s" type="%s" />' % (line.name, field_type)
 
-        def _get_field_type(line):
+        bve_field_lines = self.field_ids.filtered('view_field_type')
-            row = line.row and 'row'
+        return list(map(_get_field_def, bve_field_lines))
-            column = line.column and 'col'
-            measure = line.measure and 'measure'
-            return row or column or measure
 
-        view_fields = []
-        for line in self.line_ids:
-            def_type = _get_field_type(line)
-            if def_type:
-                view_fields.append(_get_field_def(line.name, def_type))
-        return view_fields
-
-    @api.multi
     def _create_tree_view_arch(self):
         self.ensure_one()
 
-        def _get_field_def(name):
+        def _get_field_attrs(line):
-            return """<field name="{}" />""".format(name)
+            attr = line.list_attr
+            res = attr and '%s="%s"' % (attr, line.description) or ''
+            return '<field name="%s" %s />' % (line.name, res)
 
-        view_fields = []
+        bve_field_lines = self.field_ids.filtered(lambda l: l.in_list)
-        for line in self.line_ids:
+        return list(map(_get_field_attrs, bve_field_lines))
-            if line.in_list and not line.join_node:
-                view_fields.append(_get_field_def(line.name))
-        return view_fields
 
-    @api.multi
     def _create_bve_view(self):
         self.ensure_one()
+        View = self.env['ir.ui.view'].sudo()
+
+        # delete old views
+        View.search([('model', '=', self.model_name)]).unlink()
 
         # create views
-        View = self.env['ir.ui.view']
+        View.create([{
-        old_views = View.sudo().search([('model', '=', self.model_name)])
-        old_views.unlink()
-
-        view_vals = [{
             'name': 'Pivot Analysis',
             'type': 'pivot',
             'model': self.model_name,
@@ -170,12 +221,10 @@ class BveView(models.Model):
                        {}
                        </search>
                     """.format("".join(self._create_view_arch()))
-        }]
+        }])
-
-        View.sudo().create(view_vals)
 
         # create Tree view
-        tree_view = View.sudo().create({
+        tree_view = View.create({
             'name': 'Tree Analysis',
             'type': 'tree',
             'model': self.model_name,
@@ -188,7 +237,7 @@ class BveView(models.Model):
             })
 
         # set the Tree view as the default one
-        action_vals = {
+        action = self.env['ir.actions.act_window'].sudo().create({
             'name': self.name,
             'res_model': self.model_name,
             'type': 'ir.actions.act_window',
@@ -196,17 +245,14 @@ class BveView(models.Model):
             'view_mode': 'tree,graph,pivot',
             'view_id': tree_view.id,
             'context': "{'service_name': '%s'}" % self.name,
-        }
+        })
 
-        ActWindow = self.env['ir.actions.act_window']
-        action_id = ActWindow.sudo().create(action_vals)
         self.write({
-            'action_id': action_id.id,
+            'action_id': action.id,
             'view_id': tree_view.id,
             'state': 'created'
         })
 
-    @api.multi
     def _build_access_rules(self, model):
         self.ensure_one()
 
@@ -218,97 +264,82 @@ class BveView(models.Model):
             })
         else:
             # read access only to model
-            access_vals = []
+            access_vals = [{
-            for group in self.group_ids:
+                'name': 'read access to ' + self.model_name,
-                access_vals += [{
+                'model_id': model.id,
-                    'name': 'read access to ' + self.model_name,
+                'group_id': group.id,
-                    'model_id': model.id,
+                'perm_read': True
-                    'group_id': group.id,
+            } for group in self.group_ids]
-                    'perm_read': True
-                }]
             self.env['ir.model.access'].sudo().create(access_vals)
 
-    @api.multi
     def _create_sql_view(self):
         self.ensure_one()
 
-        def get_fields_info(lines):
-            fields_info = []
-            for line in lines:
-                vals = {
-                    'table': self.env[line.field_id.model_id.model]._table,
-                    'table_alias': line.table_alias,
-                    'select_field': line.field_id.name,
-                    'as_field': line.name,
-                    'join': line.join_node,
-                }
-                fields_info.append(vals)
-            return fields_info
-
-        def get_join_nodes(info):
-            return [(
-                f['table_alias'],
-                f['join'],
-                f['select_field']
-            ) for f in info if f['join']]
-
-        def get_tables(info):
-            return set([(f['table'], f['table_alias']) for f in info])
-
-        def get_select_fields(info):
-            first_field = [(info[0]['table_alias'] + ".id", "id")]
-            next_fields = [
-                ("{}.{}".format(f['table_alias'], f['select_field']),
-                 f['as_field']) for f in info if 'join_node' not in f
-            ]
-            return first_field + next_fields
-
-        if not self.line_ids:
-            raise UserError(_('No data to process.'))
-
-        info = get_fields_info(self.line_ids)
-        select_fields = get_select_fields(info)
-        tables = get_tables(info)
-        join_nodes = get_join_nodes(info)
-
         view_name = self.model_name.replace('.', '_')
-        select_str = ', '.join(["{} AS {}".format(f[0], f[1])
+        query = self.query and self.query.replace('\n', ' ')
-                                for f in select_fields])
-        from_str = ', '.join(["{} AS {}".format(t[0], t[1])
-                              for t in list(tables)])
-        where_str = " AND ".join(["{}.{} = {}.id".format(j[0], j[2], j[1])
-                                  for j in join_nodes])
 
         # robustness in case something went wrong
         self._cr.execute('DROP TABLE IF EXISTS %s', (AsIs(view_name), ))
 
-        self.query = """
+        # create postgres view
-            SELECT %s
+        self.env.cr.execute('CREATE or REPLACE VIEW %s as (%s)', (
+            AsIs(view_name), AsIs(query), ))
 
-            FROM  %s
+    @api.depends('line_ids', 'state')
-            """ % (AsIs(select_str), AsIs(from_str), )
+    def _compute_sql_query(self):
-        if where_str:
+        for bve_view in self:
-            self.query += """
+            tables_map = {}
-            WHERE %s
+            select_str = '\n CAST(row_number() OVER () as integer) AS id'
-            """ % (AsIs(where_str), )
+            for line in bve_view.field_ids:
+                table = line.table_alias
+                select = line.field_id.name
+                as_name = line.name
+                select_str += ',\n {}.{} AS {}'.format(table, select, as_name)
 
-        self.env.cr.execute(
+                if line.table_alias not in tables_map:
-            """CREATE or REPLACE VIEW %s as (
+                    table = self.env[line.field_id.model_id.model]._table
-            %s
+                    tables_map[line.table_alias] = table
-            )""", (AsIs(view_name), AsIs(self.query), ))
+            seen = set()
+            from_str = ""
+            if not bve_view.relation_ids and bve_view.field_ids:
+                first_line = bve_view.field_ids[0]
+                table = tables_map[first_line.table_alias]
+                from_str = "{} AS {}".format(table, first_line.table_alias)
+            for line in bve_view.relation_ids:
+                table = tables_map[line.table_alias]
+                table_format = "{} AS {}".format(table, line.table_alias)
+                if not from_str:
+                    from_str += table_format
+                    seen.add(line.table_alias)
+                if line.table_alias not in seen:
+                    seen.add(line.table_alias)
+                    from_str += "\n"
+                    from_str += " LEFT" if line.left_join else ""
+                    from_str += " JOIN {} ON {}.id = {}.{}".format(
+                        table_format,
+                        line.join_node, line.table_alias, line.field_id.name
+                    )
+                if line.join_node not in seen:
+                    from_str += "\n"
+                    seen.add(line.join_node)
+                    from_str += " LEFT" if line.left_join else ""
+                    from_str += " JOIN {} AS {} ON {}.{} = {}.id".format(
+                        tables_map[line.join_node], line.join_node,
+                        line.table_alias, line.field_id.name, line.join_node
+                    )
+            bve_view.query = """SELECT %s\n\nFROM %s
+                """ % (AsIs(select_str), AsIs(from_str),)
 
-    @api.multi
     def action_translations(self):
         self.ensure_one()
         if self.state != 'created':
             return
-        model = self.env['ir.model'].sudo().search([
+        self = self.sudo()
-            ('model', '=', self.model_name)
+        model = self.env['ir.model'].search([('model', '=', self.model_name)])
-        ])
+        IrTranslation = self.env['ir.translation']
-        IrTranslation = self.env['ir.translation'].sudo()
         IrTranslation.translate_fields('ir.model', model.id)
-        for field_id in model.field_id.ids:
+        for field in model.field_id:
-            IrTranslation.translate_fields('ir.model.fields', field_id)
+            IrTranslation.translate_fields('ir.model.fields', field.id)
         return {
             'name': 'Translations',
             'res_model': 'ir.translation',
@@ -328,37 +359,10 @@ class BveView(models.Model):
             ],
         }
 
-    @api.multi
     def action_create(self):
         self.ensure_one()
 
-        def _prepare_field(line):
+        # consistency checks
-            field = line.field_id
-            vals = {
-                'name': line.name,
-                'complete_name': field.complete_name,
-                'model': self.model_name,
-                'relation': field.relation,
-                'field_description': line.description,
-                'ttype': field.ttype,
-                'selection': field.selection,
-                'size': field.size,
-                'state': 'manual',
-                'readonly': True,
-                'groups': [(6, 0, field.groups.ids)],
-            }
-            if vals['ttype'] == 'monetary':
-                vals.update({'ttype': 'float'})
-            if field.ttype == 'selection' and not field.selection:
-                model_obj = self.env[field.model_id.model]
-                selection = model_obj._fields[field.name].selection
-                if callable(selection):
-                    selection_domain = selection(model_obj)
-                else:
-                    selection_domain = selection
-                vals.update({'selection': str(selection_domain)})
-            return vals
-
         self._check_invalid_lines()
         self._check_groups_consistency()
 
@@ -369,13 +373,12 @@ class BveView(models.Model):
         self._create_sql_view()
 
         # create model and fields
-        fields_data = self.line_ids.filtered(lambda l: not l.join_node)
+        bve_fields = self.line_ids.filtered(lambda l: not l.join_node)
-        field_ids = [(0, 0, _prepare_field(f)) for f in fields_data]
         model = self.env['ir.model'].sudo().with_context(bve=True).create({
             'name': self.name,
             'model': self.model_name,
             'state': 'manual',
-            'field_id': field_ids,
+            'field_id': [(0, 0, f) for f in bve_fields._prepare_field_vals()],
         })
 
         # give access rights
@@ -417,11 +420,14 @@ class BveView(models.Model):
 
     def _check_invalid_lines(self):
         self.ensure_one()
+        if not self.line_ids:
+            raise ValidationError(_('No data to process.'))
+
         if any(not line.model_id for line in self.line_ids):
             invalid_lines = self.line_ids.filtered(lambda l: not l.model_id)
             missing_models = set(invalid_lines.mapped('model_name'))
             missing_models = ', '.join(missing_models)
-            raise UserError(_(
+            raise ValidationError(_(
                 'Following models are missing: %s.\n'
                 'Probably some modules were uninstalled.' % (missing_models,)
             ))
@@ -429,11 +435,10 @@ class BveView(models.Model):
             invalid_lines = self.line_ids.filtered(lambda l: not l.field_id)
             missing_fields = set(invalid_lines.mapped('field_name'))
             missing_fields = ', '.join(missing_fields)
-            raise UserError(_(
+            raise ValidationError(_(
                 'Following fields are missing: %s.' % (missing_fields,)
             ))
 
-    @api.multi
     def open_view(self):
         self.ensure_one()
         self._check_invalid_lines()
@@ -447,7 +452,6 @@ class BveView(models.Model):
         default = dict(default or {}, name=_("%s (copy)") % self.name)
         return super().copy(default=default)
 
-    @api.multi
     def action_reset(self):
         self.ensure_one()
 
@@ -479,7 +483,6 @@ class BveView(models.Model):
         if has_menus:
             return {'type': 'ir.actions.client', 'tag': 'reload'}
 
-    @api.multi
     def unlink(self):
         if self.filtered(lambda v: v.state == 'created'):
             raise UserError(
@@ -490,7 +493,7 @@ class BveView(models.Model):
     @api.model
     def _sync_lines_and_data(self, data):
         line_ids = [(5, 0, 0)]
-        fields_info = {}
+        fields_info = []
         if data:
             fields_info = json.loads(data)
 
@@ -524,6 +527,7 @@ class BveView(models.Model):
 
     @api.constrains('line_ids')
     def _constraint_line_ids(self):
+        models_with_tables = self.env.registry.models.keys()
         for view in self:
             nodes = view.line_ids.filtered(lambda n: n.join_node)
             nodes_models = nodes.mapped('table_alias')
@@ -535,17 +539,20 @@ class BveView(models.Model):
                 raise ValidationError(err_msg)
             if len(set(not_nodes_models) - set(nodes_models)) > 1:
                 raise ValidationError(err_msg)
+            models = view.line_ids.mapped('model_id')
+            if models.filtered(lambda m: m.model not in models_with_tables):
+                raise ValidationError(_('Abstract models not supported.'))
 
     @api.model
     def get_clean_list(self, data_dict):
         serialized_data = json.loads(data_dict)
         table_alias_list = set()
         for item in serialized_data:
-            if item.get('join_node', -1) == -1:
+            if item.get('join_node', -1) in [-1, False]:
                 table_alias_list.add(item['table_alias'])
 
         for item in serialized_data:
-            if item.get('join_node', -1) != -1:
+            if item.get('join_node', -1) not in [-1, False]:
                 if item['table_alias'] not in table_alias_list:
                     serialized_data.remove(item)
                 elif item['join_node'] not in table_alias_list:

bi_view_editor/models/bve_view_line.py

@@ -22,42 +22,94 @@ class BveViewLine(models.Model):
     description = fields.Char(translate=True)
     relation = fields.Char()
     join_node = fields.Char()
+    left_join = fields.Boolean()
 
     row = fields.Boolean()
     column = fields.Boolean()
     measure = fields.Boolean()
     in_list = fields.Boolean()
+    list_attr = fields.Selection([
+        ('sum', 'Sum'),
+        ('avg', 'Average'),
+    ], string='List Attribute', default='sum')
+    view_field_type = fields.Char(compute='_compute_view_field_type')
+
+    @api.depends('row', 'column', 'measure')
+    def _compute_view_field_type(self):
+        for line in self:
+            row = line.row and 'row'
+            column = line.column and 'col'
+            measure = line.measure and 'measure'
+            line.view_field_type = row or column or measure
 
     @api.constrains('row', 'column', 'measure')
     def _constrains_options_check(self):
         measure_types = ['float', 'integer', 'monetary']
-        for line in self:
+        for line in self.filtered(lambda l: l.row or l.column):
-            if line.row or line.column:
+            if line.join_model_id or line.ttype in measure_types:
-                if line.join_model_id or line.ttype in measure_types:
+                err_msg = _('This field cannot be a row or a column.')
-                    err_msg = _('This field cannot be a row or a column.')
+                raise ValidationError(err_msg)
-                    raise ValidationError(err_msg)
+        for line in self.filtered(lambda l: l.measure):
-            if line.measure:
+            if line.join_model_id or line.ttype not in measure_types:
-                if line.join_model_id or line.ttype not in measure_types:
+                err_msg = _('This field cannot be a measure.')
-                    err_msg = _('This field cannot be a measure.')
+                raise ValidationError(err_msg)
-                    raise ValidationError(err_msg)
+
+    @api.constrains('table_alias', 'field_id')
+    def _constrains_unique_fields_check(self):
+        seen = set()
+        for line in self.mapped('bve_view_id.field_ids'):
+            if (line.table_alias, line.field_id.id, ) not in seen:
+                seen.add((line.table_alias, line.field_id.id, ))
+            else:
+                raise ValidationError(_('Field %s/%s is duplicated.\n'
+                                        'Please remove the duplications.') % (
+                    line.field_id.model, line.field_id.name
+                ))
 
     @api.depends('field_id', 'sequence')
     def _compute_name(self):
-        for line in self:
+        for line in self.filtered(lambda l: l.field_id):
-            if line.field_id:
+            field_name = line.field_id.name
-                field_name = line.field_id.name
+            line.name = 'x_bve_%s_%s' % (line.table_alias, field_name,)
-                line.name = 'x_bve_%s_%s' % (line.sequence, field_name,)
 
     @api.depends('model_id')
     def _compute_model_name(self):
-        for line in self:
+        for line in self.filtered(lambda l: l.model_id):
-            if line.model_id:
+            line.model_name = line.model_id.model
-                line.model_name = line.model_id.model
 
     @api.depends('field_id')
     def _compute_model_field_name(self):
+        for line in self.filtered(lambda l: l.field_id):
+            field_name = line.description
+            model_name = line.model_name
+            line.field_name = '%s (%s)' % (field_name, model_name, )
+
+    def _prepare_field_vals(self):
+        vals_list = []
         for line in self:
-            if line.field_id:
+            field = line.field_id
-                field_name = line.description
+            vals = {
-                model_name = line.model_name
+                'name': line.name,
-                line.field_name = '%s (%s)' % (field_name, model_name, )
+                'complete_name': field.complete_name,
+                'model': line.bve_view_id.model_name,
+                'relation': field.relation,
+                'field_description': line.description,
+                'ttype': field.ttype,
+                'selection': field.selection,
+                'size': field.size,
+                'state': 'manual',
+                'readonly': True,
+                'groups': [(6, 0, field.groups.ids)],
+            }
+            if vals['ttype'] == 'monetary':
+                vals.update({'ttype': 'float'})
+            if field.ttype == 'selection' and not field.selection:
+                model_obj = self.env[field.model_id.model]
+                selection = model_obj._fields[field.name].selection
+                if callable(selection):
+                    selection_domain = selection(model_obj)
+                else:
+                    selection_domain = selection
+                vals.update({'selection': str(selection_domain)})
+            vals_list.append(vals)
+        return vals_list

bi_view_editor/models/ir_model.py

@@ -6,8 +6,6 @@ from collections import defaultdict
 from odoo import api, models, registry
 
 NO_BI_MODELS = [
-    'temp.range',
-    'account.statement.operation.template',
     'fetchmail.server'
 ]
 
@@ -84,13 +82,6 @@ class IrModel(models.Model):
                 model['model'], 'read', False)
         return False
 
-    @api.model
-    def sort_filter_models(self, models_list):
-        res = sorted(
-            filter(self._filter_bi_models, models_list),
-            key=lambda x: x['name'])
-        return res
-
     def get_model_list(self, model_table_map):
         if not model_table_map:
             return []
@@ -131,10 +122,7 @@ class IrModel(models.Model):
         return relation_list
 
     @api.model
-    def get_related_models(self, model_table_map):
+    def _get_related_models_domain(self, model_table_map):
-        """ Return list of model dicts for all models that can be
-            joined with the already selected models.
-        """
         domain = [('transient', '=', False)]
         if model_table_map:
             model_list = self.get_model_list(model_table_map)
@@ -144,7 +132,15 @@ class IrModel(models.Model):
             relations = [f['relation'] for f in model_list]
             domain += [
                 '|', ('id', 'in', model_ids), ('model', 'in', relations)]
-        return self.sudo().search(domain)
+        return domain
+
+    @api.model
+    def get_related_models(self, model_table_map):
+        """ Return list of model dicts for all models that can be
+            joined with the already selected models.
+        """
+        domain = self._get_related_models_domain(model_table_map)
+        return self.sudo().search(domain, order='name asc')
 
     @api.model
     def get_models(self, table_model_map=None):
@@ -155,10 +151,13 @@ class IrModel(models.Model):
         for k, v in (table_model_map or {}).items():
             model_table_map[v].append(k)
 
-        models_list = []
+        models = self.get_related_models(model_table_map)
-        for model in self.get_related_models(model_table_map):
+
-            models_list.append(dict_for_model(model))
+        # filter out abstract models (they do not have DB tables)
-        return self.sort_filter_models(models_list)
+        non_abstract_models = self.env.registry.models.keys()
+        models = models.filtered(lambda m: m.model in non_abstract_models)
+
+        return list(map(dict_for_model, models))
 
     @api.model
     def get_join_nodes(self, field_data, new_field):
@@ -167,26 +166,6 @@ class IrModel(models.Model):
             Return all possible join nodes to add new_field to the query
             containing model_ids.
         """
-        def _get_model_table_map(field_data):
-            table_map = defaultdict(list)
-            for data in field_data:
-                table_map[data['model_id']].append(data['table_alias'])
-            return table_map
-
-        def _get_join_nodes_dict(model_table_map, new_field):
-            join_nodes = []
-            for alias in model_table_map[new_field['model_id']]:
-                join_nodes.append({'table_alias': alias})
-
-            for field in self.get_model_list(model_table_map):
-                if new_field['model'] == field['relation']:
-                    join_nodes.append(field)
-
-            for field in self.get_relation_list(model_table_map):
-                if new_field['model_id'] == field['model_id']:
-                    join_nodes.append(field)
-            return join_nodes
-
         def remove_duplicate_nodes(join_nodes):
             seen = set()
             nodes_list = []
@@ -198,40 +177,44 @@ class IrModel(models.Model):
             return nodes_list
 
         self = self.with_context(lang=self.env.user.lang)
-        model_table_map = _get_model_table_map(field_data)
-        keys = [(field['table_alias'], field['id'])
-                for field in field_data if field.get('join_node', -1) != -1]
-        join_nodes = _get_join_nodes_dict(model_table_map, new_field)
-        join_nodes = remove_duplicate_nodes(join_nodes)
 
-        return list(filter(
+        keys = []
-            lambda x: 'id' not in x or
+        model_table_map = defaultdict(list)
-                      (x['table_alias'], x['id']) not in keys, join_nodes))
+        for field in field_data:
+            model_table_map[field['model_id']].append(field['table_alias'])
+            if field.get('join_node', -1) != -1:
+                keys.append((field['table_alias'], field['id']))
+
+        # nodes in current model
+        existing_aliases = model_table_map[new_field['model_id']]
+        join_nodes = [{'table_alias': alias} for alias in existing_aliases]
+
+        # nodes in past selected models
+        for field in self.get_model_list(model_table_map):
+            if new_field['model'] == field['relation']:
+                if (field['table_alias'], field['id']) not in keys:
+                    join_nodes.append(field)
+
+        # nodes in new model
+        for field in self.get_relation_list(model_table_map):
+            if new_field['model_id'] == field['model_id']:
+                if (field['table_alias'], field['id']) not in keys:
+                    join_nodes.append(field)
+
+        return remove_duplicate_nodes(join_nodes)
 
     @api.model
     def get_fields(self, model_id):
         self = self.with_context(lang=self.env.user.lang)
-        domain = [
+
+        fields = self.env['ir.model.fields'].sudo().search([
             ('model_id', '=', model_id),
             ('store', '=', True),
             ('name', 'not in', models.MAGIC_COLUMNS),
             ('ttype', 'not in', NO_BI_TTYPES)
-        ]
+        ], order='field_description desc')
-        fields_dict = []
+        fields_dict = list(map(dict_for_field, fields))
-        for field in self.env['ir.model.fields'].sudo().search(domain):
+        return fields_dict
-            fields_dict.append({
-                'id': field.id,
-                'model_id': model_id,
-                'name': field.name,
-                'description': field.field_description,
-                'type': field.ttype,
-                'model': field.model,
-            })
-        return sorted(
-            fields_dict,
-            key=lambda x: x['description'],
-            reverse=True
-        )
 
     @api.model
     def create(self, vals):

bi_view_editor/readme/CREDITS.rst

@@ -1,11 +1,6 @@
-Images
-------
-
-* Odoo Community Association: `Icon <https://github.com/OCA/maintainer-tools/blob/master/template/module/static/description/icon.svg>`_.
-
 Funders
 -------
 
-The development of this module has been financially supported by:
+The development of this module for Odoo 11.0 has been financially supported by:
 
 * IDEAL Connaissances SAS https://www.idealconnaissances.com

bi_view_editor/readme/INSTALL.rst

@@ -0,0 +1,18 @@
+In the Odoo configuration file add ``bi_view_editor`` in the list
+``server_wide_modules``:
+
+.. code-block:: ini
+
+  [options]
+  (...)
+  server_wide_modules = web,bi_view_editor
+  (...)
+
+Alternatively specify ``--load=bi_view_editor`` when starting Odoo by command line.
+
+Optionally it is possible to enable the view of the ER Diagram. For this you
+need to install `Graphviz`, an open source graph visualization software:
+
+.. code-block:: bash
+
+   ``sudo apt-get install graphviz``

bi_view_editor/readme/ROADMAP.rst

@@ -1,9 +1,7 @@
-* Non-stored fields and many2many fields are not supported
+* Non-stored fields and many2many fields are not supported.
-* Provide graph view for table relations
+* Provide a tutorial (eg. a working example of usage).
-* Extend the capabilities of the tree views (e.g. add sums)
+* Find better ways to extend the *_auto_init()* without override.
-* Provide a tutorial (eg. a working example of usage)
+* Possibly avoid the monkey patches.
-* Implement a more advanced UI, with possibilities to use LEFT JOIN as default instead of INNER JOIN
+* Data the user has no access to (e.g. in a multi company situation) can be
-* Find better ways to extend the *_auto_init()* without override
+  viewed by making a view. Would be nice if models available to select when
-* Possibly avoid the monkey patches
+  creating a view are limited to the ones that have intersecting groups.
-* Data the user has no access to (e.g. in a multi company situation) can be viewed by making a view
-* Would be nice if models available to select when creating a view are limited to the ones that have intersecting groups (for non technical users)

bi_view_editor/readme/USAGE.rst

@@ -1,10 +1,23 @@
 To graphically design your analysis data-set:
 
 - From the Dashboards menu, select "Custom BI Views"
-- Browse trough the business objects in the Query tab
+- Browse trough the business objects in the "Query Builder" tab
 - Pick the interesting fields (Drag & Drop)
-- For each selected field, right-click on the Options column and select whether it's a row, column or measure; if you want to remove the field from the list view, unflag the checkbox ´List´ in the Options column
+- For each selected field, right-click on the Options column and select whether
+  it's a row, column or measure; if you want to remove the field from the list
+  view, unflag the checkbox ´List´ in the Options column
 - Save and click "Generate BI View"
 - Click "Open BI View" to view the result
-- If module Dashboard (board) is installed, the standard "Add to My Dashboard" functionality would be available
+
-- Click "Create a menu" to create a new menu item directly linked to your new BI view (this feature is available in developer mode); when the BI view is reset back to draft this menu will be removed, and you will need to re-create the menu entry.
+To access the created BI View with a dedicated menu:
+
+- If module Dashboard (board) is installed, the standard "Add to My Dashboard"
+  functionality would be available
+- Click "Create a menu" to create a new menu item directly linked to your new
+  BI view (this feature is available in developer mode); when the BI view is
+  reset back to draft this menu will be removed, and you will need to re-create
+  the menu entry.
+
+A more advanced UI is also available under the "Details" tab. It provides extra
+possibilities for more advanced users, like to use LEFT JOIN instead of the
+default INNER JOIN.

bi_view_editor/static/src/js/bi_view_editor.js

@@ -96,7 +96,7 @@ odoo.define('bi_view_editor', function (require) {
         getTableAlias: function (field) {
             if (typeof field.table_alias === 'undefined') {
                 var model_ids = this.field_list.getModelIds();
-                var n = 0;
+                var n = 1;
                 while (typeof model_ids["t" + n] !== 'undefined') {
                     n++;
                 }

bi_view_editor/tests/test_bi_view.py

@@ -5,7 +5,7 @@ import json
 
 import odoo
 from odoo.tests.common import TransactionCase
-from odoo.exceptions import UserError
+from odoo.exceptions import UserError, ValidationError
 from ..hooks import post_load, uninstall_hook
 
 
@@ -179,9 +179,10 @@ class TestBiViewEditor(TransactionCase):
         }
         bi_view4 = self.env['bve.view'].create(vals)
         self.assertEqual(len(bi_view4), 1)
+        self.assertTrue(bi_view4.er_diagram_image)
 
         # create sql view
-        with self.assertRaises(UserError):
+        with self.assertRaises(ValidationError):
             bi_view4.action_create()
 
     def test_08_get_models(self):
@@ -200,6 +201,7 @@ class TestBiViewEditor(TransactionCase):
         bi_view = self.env['bve.view'].create(vals)
         self.assertEqual(len(bi_view), 1)
         self.assertEqual(len(bi_view.line_ids), 3)
+        self.assertTrue(bi_view.er_diagram_image)
 
         # check lines
         line1 = bi_view.line_ids[0]
@@ -328,7 +330,7 @@ class TestBiViewEditor(TransactionCase):
         for line in bi_view1.line_ids:
             self.assertFalse(line.model_id)
             self.assertTrue(line.model_name)
-        with self.assertRaises(UserError):
+        with self.assertRaises(ValidationError):
             bi_view1.action_create()
 
     def test_14_check_lines_missing_fieldl(self):
@@ -347,7 +349,7 @@ class TestBiViewEditor(TransactionCase):
         for line in bi_view1.line_ids:
             self.assertFalse(line.field_id)
             self.assertTrue(line.field_name)
-        with self.assertRaises(UserError):
+        with self.assertRaises(ValidationError):
             bi_view1.action_create()
 
     def test_15_create_lines(self):

bi_view_editor/views/bve_view.xml

@@ -43,33 +43,50 @@
                         <field name="name" attrs="{'readonly': [('state','=','created')]}" colspan="4"/>
                     </h1>
                     <notebook>
-                        <page string="Query">
+                        <page string="Query Builder">
                             <group>
                                 <field name="data" widget="BVEEditor" nolabel="1" attrs="{'readonly': [('state','=','created')]}"/>
                             </group>
                         </page>
-                        <page string="Lines" groups="base.group_no_one">
+                        <page string="ER Diagram" attrs="{'invisible': [('er_diagram_image','=',False)]}">
                             <group>
-                                <field name="line_ids" nolabel="1" attrs="{'readonly': [('state','=','created')]}">
+                                <field nolabel="1" name="er_diagram_image" widget="image"/>
-                                    <tree decoration-info="join_model_id" editable="bottom">
+                            </group>
+                        </page>
+                        <page string="Details">
+                            <group>
+                                <field name="field_ids" attrs="{'readonly': [('state','=','created')]}">
+                                    <tree editable="bottom" decoration-muted="in_list == False">
                                         <field name="sequence" widget="handle"/>
                                         <field name="description" string="Field"/>
-                                        <field name="model_id"/>
+                                        <field name="model_id" readonly="1"/>
                                         <field name="table_alias"/>
-                                        <field name="join_model_id"/>
-                                        <field name="join_node"/>
                                         <field name="ttype" invisible="1"/>
-                                        <field name="row" widget="toggle_button" attrs="{'invisible': ['|', ('join_model_id','!=',False), ('ttype','in',('float', 'integer', 'monetary'))]}"/>
+                                        <field name="row" widget="toggle_button" attrs="{'invisible': [('ttype','in',('float', 'integer', 'monetary'))]}"/>
-                                        <field name="column" widget="toggle_button" attrs="{'invisible': ['|', ('join_model_id','!=',False), ('ttype','in',('float', 'integer', 'monetary'))]}"/>
+                                        <field name="column" widget="toggle_button" attrs="{'invisible': [('ttype','in',('float', 'integer', 'monetary'))]}"/>
-                                        <field name="measure" widget="toggle_button" attrs="{'invisible': ['|', ('join_model_id','!=',False), ('ttype','not in',('float', 'integer', 'monetary'))]}"/>
+                                        <field name="measure" widget="toggle_button" attrs="{'invisible': [('ttype','not in',('float', 'integer', 'monetary'))]}"/>
-                                        <field name="in_list" widget="boolean_toggle" attrs="{'invisible': [('join_model_id','!=',False)]}"/>
+                                        <field name="in_list" widget="boolean_toggle"/>
+                                        <field name="list_attr" attrs="{'invisible': ['|',('in_list','=',False),('ttype','not in',('float', 'integer'))]}"/>
+                                    </tree>
+                                </field>
+                            </group>
+                            <group>
+                                <field name="relation_ids" attrs="{'readonly': [('state','=','created')]}">
+                                    <tree editable="bottom">
+                                        <field name="sequence" widget="handle"/>
+                                        <field name="description" string="Field"/>
+                                        <field name="model_id" readonly="1"/>
+                                        <field name="table_alias"/>
+                                        <field name="join_model_id" readonly="1"/>
+                                        <field name="join_node"/>
+                                        <field name="left_join" widget="toggle_button"/>
                                     </tree>
                                 </field>
                             </group>
                         </page>
-                        <page string="SQL" groups="base.group_no_one" attrs="{'invisible': [('state','!=','created')]}">
+                        <page string="SQL" groups="base.group_no_one">
                             <group>
-                                <field name="query" nolabel="1" readonly="1"/>
+                                <field name="query" nolabel="1" />
                             </group>
                         </page>
                         <page string="Security">
@@ -86,7 +103,6 @@
 
     <record id="action_bi_view_editor_view_form" model="ir.actions.act_window">
         <field name="name">Custom BI Views</field>
-        <field name="type">ir.actions.act_window</field>
         <field name="res_model">bve.view</field>
         <field name="view_type">form</field>
         <field name="view_mode">tree,form</field>

bi_view_editor/wizard/wizard_ir_model_menu_create.py

@@ -7,7 +7,6 @@ from odoo import api, models
 class WizardModelMenuCreate(models.TransientModel):
     _inherit = 'wizard.ir.model.menu.create'
 
-    @api.multi
     def menu_create(self):
         if self.env.context.get('active_model') == 'bve.view':
             self.ensure_one()