Web page generation

Dynamic html generation

The creation of web pages builds on the capabilities of the Python Flask and jinja libraries. The handlers for each webpage are typically within the views.py scripts in various directories of entero, with each directory relating to the different broad areas of enterobase functionaility. The handlers are prefixed with the @XXXX.route prefix, such as:

@main.route('/')
def index():

where XXXX corresponds to the different groups of URLs such as main, species etc, and where the handlers are then found in the XXXX/views.py python scripts. The prefixes indicate to flask which handlers to use for which URLs. The html pages are then rendered using code such as:

db_info = format_db_info(db_info)
return render_template('index.html',db_info=db_info, total_strains = total_strains, help = app.config['WIKI_BASE'])

which takes the fixed template html file from entero/templates directory and makes appropriate substitutions before returning the completed webpage to the client for display.

The templates are a mixture of html, embedded python which is executed during the process of generating the web page,(enclosed in {% … %} ) and tags for substitution (enclosed in {{…}}) as in the following example, where the db_info python array passed into render_template is cycled through creating a new block of html populated with the contents (rec[‘name’] etc)

The templates are arranged in a heirarchy such that features that are present in multiple web pages can be defined in a common ancester template and inherited. Note that any template blocks, such as {% block tail %} that are used by child templates must be defined in the root template. The root template for Enterobase is base.html.

Template Hierarchy

The template hierarchy is:

• base.html

  • species/species_base.html

    • species/species.html - The home page for species

  • bsl_base.html - includes side_bar.html

    • species/species_index.html
    • species/allele_st_query
    • species/download_7_gene.html
    • admin/email_users.html
    • admin/download_genome.html

  • species/span_tree.html

  • jql_base.html

    • split_table_base.html - Wrapper for display of data in a grid

      • species/view_edit_strains/edit_strain_metadata.html - For seraching for and viewing strain data

    • species/
    • admin/jobs_view.html - For admin view of all jobs

  • generic_form.html
  • admin/index.html - Base for administrative functions

• swagger-ui.html

Tables within web pages

The displaying of lists of strains, their metadata and the experimental data associated with schemes is done within the edit_strain_metadata.html template pages using a set of javascript files that are found in the entero/static/js/table directory. These inherit from the editablegrid-2.0.1.js (see https://www.editablegrid.net/en/)

• editablegrid

  • validation_grid (includes generic support for extra_row_info)

    • base_experiment_grid (SeroPred)

      • base_scheme_grid

        • scheme_grid (7 gene scheme)
        • medium_scheme_grid (cgMLST, wgMLST,rMLST)

      • amr_grid
      • annotation_grid (annotation)
      • assembly_grid (assembly stats)
      • custom_view_grid
      • experiment_grid
      • hierarchy_grid
      • ugs_grid

    • scheme_validation_grid
    • buddy_grid
    • jobs_grid (jobs_monitor)
    • locus_grid
    • reads_grid

    • strain_validation_grid

      • sra_validation_grid

        • uber_strain_grid (for LH metadata screen)

      • upload_reads_validation_grid

The choice of which grid is used to display whcih experimental data is made by the js_grid parameter in the param column of the scheme’s entry in the schemes table. The js_grid column appears to be ignored, indeed its entry in generic_models.py is commented out. The data to be displayed is generated on the server by the code in ExtraFuncs/query_functions.py

strain_validation_grid and functional inheritance

This is a good example of the way that specific functionaility is implemented by each generation of the class hierarchy. The common information to be displayed when adding metadata for uploading reads and displaying strains is held in strain_validation_grid.js. When the strains are displayed on the GUI there is additional information associated with the data retrieved from NCBI for SRA strains, which is handled by sra_validation_grid.js and finally the specific functionality associated with uberstrains is in uber_strain_grid.js

javascript

The webpages make extensive use of javascript files, which are loaded from entero/static/js. Each entry in the html for loading them tends to be suffixed with ‘version={{config[‘JAVASCRIPT_VERSION’]}}’, which is substituted with the current value of config[‘JAVASCRIPT_VERSION’] when the template html file is loaded. This config value is set to a (new) random number within config.py each time the enterobase-web is restarted, which ensures that the javascript files are reloaded rather than a cached version used. This is particularly useful when testing changes to the contents of these files.

Support of ‘eyes’ for extra info

The ‘eye’ at the left of some experimental data panes appears if there is an “extra_row_info” element for the row in that data that is returned from the server. This results in validation_grid.js code copying the extra_row_info into the [‘extra_row_info’] dictionary element for the record. It is then necessary to add an ‘addExtraRenderer’ entry in the js file for the specific data (e.g. in amr_grid.js) that displays the eye if the extra_row_info is present. It is also necessary to have a grid_data.push method that creates the column in the grid for the eye. Finally a addCustomColumnHandler method is requried for the column that performs the action associated with the clicking the eye. In general the amr_grid.js provides a good working example of the implementation of the eye function.

Displaying of Strain and Experimental data

This is catered for with the search_strains.html template which displays the search dialog and the the dual pane display of the results. This uses uber_strain_grid.js to display the lefthand ‘strain’ pane, and the appropriate variant of base_experiment_grid.js to display the selected experimental data. When loading the uber_strain_grid, the following call within strain_validation_grid.js loads up the information about the columns to be displayed

Enterobase.call_restful_json("/species/"+this.species+"/get_strain_columns"

When initially loading search_strains.html, the following call

Enterobase.call_restful_json("/get_experiment_details"

uses databases/database.py:get_experiment_details to get information about all the schems and associated fields which are used to populate both the drop down menu options in the search box and the columns in the left and right hand panes for displaying the results. The information about the columns in the display panes is configured using the data in the data_params table, see Configuration info for metadata and results fields.

Support of Admin screens

For admin screens, some of the screens inherit from the templates that are provided by the flask_admin python library and are contained in the flask_admin/templates directory of the phython library. The admin templates are loaded by the code in entero/admin/views.py, The code inherits from the ModelView class in flask_admin/contrib/sqla/view.py which prevides generic code for viewing data that is held in an sql database, making extensive use of the SQLalchemy support for databases. There are currently three releases of the templates in the flask_admin library, bootstrap2,3 and 4, and the code makes reference to the bootstrap2 static files as can be seen in, for example, the lib.form_css section of entero/templates/admin/admin_list.html. Note that the URL used is, for example, “/admin/static/bootstrap/bootstrap2/ and the flask_admin code redirects requests for such files to flask_admin/static/bootstrap/bootstrap2.