Tag Archives: SQL Developer

Easy Audit Data Analysis with SQL Developer Reports

In one of my last blog post SQL Toolbox for simplified Oracle Unified Audit Data Analysis, I introduced a set of scripts designed to streamline Oracle Unified Audit data analysis. These scripts, now available on my GitHub repository oehrlis/oradba, have received positive feedback. Building on that, I’d like to explore an alternative approach to augment your audit data analysis process with SQL Developer Reports.

SQL Plus is available on any database server, so you can quickly perform an initial analysis of audit data using the SQL scripts presented without any additional requirements. It should be noted, however, that while SQLPlus-based scripts are valuable tools, they are not always perfectly suited to all analytical requirements. In certain scenarios, more graphical analysis and the ability to drill down on specific data points are more helpful. This also includes simply getting an overview and drilling down in some cases. To fulfill these differentiated requirements, I use Oracle SQL Developer, a versatile tool that offers the ability to create custom reports. These reports provide a graphical and interactive way to analyze your audit data. In this blog post, I will provide an overview of the reports available and briefly explain how you can seamlessly integrate them into your audit data analysis workflow.

SQL Developer Reports

The Unified Audit Asessment Reprorts for SQL Developers are divided into the following categories:

  • Audit Charts A series of audit reports with diagrams and drill-downs, e.g. audit events per day and hour. These reports are ideal for an initial visual overview of the audit events. However, the runtime of the queries depends heavily on the amount of audit data. In some cases, the reports can be limited to a few days.
  • Audit Configuration Reports on audit configuration, memory usage, segments, audit policies, cleanup events, cleanup jobs and more. These reports provide a good overview of what is configured and how much audit data is already available.
  • Audit Sessions Overview of the various audit sessions with drill-down to the individual events of a session. Reports are available for the various audit types, e.g. proxy session, standard, RMAN, DataPump and more. These reports are ideal for checking what exactly was executed within a single session.
  • Generate Statements These are not really reports as such, but queries to generate the various statements for creating, deleting, enabling or disabling the audit policies. Corresponding authorization on DBMS_METADATA is required.
  • Miscellaneous All types of audit reports that have not yet been categorized. Mainly things that are still under development.
  • Top Audit Events Provide information about audit events that are selected and organized according to various criteria. This information can help you to identify audit events that occur more frequently than expected or generate more data than expected.

The queries were created to the best of our knowledge and belief. Nevertheless, the individual queries may take a little more time, depending on the amount of data. If necessary, you can copy the reports and optimize / modify them accordingly. Suggestions for improvements are of course always welcome.

Audit Reports in Detail

Audit Charts

ReportDescription
Audit Events by DayChart for the number of audit events per day with the option to drill down by audit events per hour.
Audit Events by UserChart for the number of audit events per database user with the option to drill down audit events by audit policy, action, user host and audit policy with actions.
Audit Chart Reports
Audit events by day with drill-down by hour

Audit Configuration

ReportDescription
Storage UsageThis report provides comprehensive details about the usage and configuration of the audit store, focusing especially on the DB ID and the archiving timestamp. It enables you to assess whether audit data can be deleted based on its age or if it originates from a different database. Additionally, the report offers a drill-down feature to view partition information, storage details, and purge statements. Similar to sdua_usage.sql, sdua_stostm.sql and sdua_prgstm.sql.
Clean up EventsDisplays the audit cleanup event from DBA_AUDIT_MGMT_CLEAN_EVENTS.
Clean up JobsDisplays the audit cleanup jobs from DBA_AUDIT_MGMT_CLEANUP_JOBS
ConfigurationDisplays the audit configuration from DBA_AUDIT_MGMT_CONFIG_PARAMS
Last Archive TimestampDisplays the audit archive timestamp from DBA_AUDIT_MGMT_LAST_ARCH_TS
Unified Audit PoliciesThis report offers a comprehensive overview of unified audit policies, drawing data from the views AUDIT_UNIFIED_POLICIES and AUDIT_UNIFIED_ENABLED_POLICIES. It facilitates a deeper understanding of the configuration and current status of audit policies within the system, similar to the script saua_pol.sql. The report includes extensive drill-down options, allowing for detailed analysis based on comments, actions, and statements such as AUDIT, NOAUDIT, CREATE, and DROP, as well as events and inactive policies.
Audit Configuration Reports
Audit Storage Usage

Audit Sessions

ReportDescription
by any typeDisplays audit sessions for any audit type, similar in function to the script saua_as.sql. With drill-down to session details.
by type DataPumpDisplays audit sessions for audit type DataPump, similar in function to the script saua_asdp.sql. With drill-down to session details.
by type DB VaultDisplays audit sessions for audit type Database Vault, similar in function to the script saua_asdbv.sql. With drill-down to session details.
by type Direct path APIDisplays audit sessions for audit type Direct path API based on UNIFIED_AUDIT_TRAIL. With drill-down to session details.
by type FGADisplays audit sessions for audit type FGA, similar in function to the script saua_asfga.sql. With drill-down to session details
by type ProtocolDisplays audit sessions for audit type Protocol. With drill-down to session details.
by type RMAN_AUDITDisplays audit sessions for audit type RMAN_AUDIT, similar in function to the script saua_asbck.sql. With drill-down to session details.
by type StandardDisplays audit sessions for audit type standard, similar in function to the script saua_asstd.sql. With drill-down to session details.
Proxy SessionsDisplays audit proxy sessions for audit type Standard based on UNIFIED_AUDIT_TRAIL. With drill-down to session details.
Session DetailsList detail of a particular session.
Session OverviewDisplays audit sessions for any audit type, similar in function to the script saua_as.sql.
Audit Session Reports

Generate Statements

ReportDescription
Create all audit policiesGenerates statements to recreate all audit policies as currently configured in AUDIT_UNIFIED_ENABLED_POLICIES.
Disable all audit policiesGenerates statements to disable all audit policies as currently configured in AUDIT_UNIFIED_ENABLED_POLICIES.
Drop all audit policiesGenerates statements to remove all audit policies, excluding those maintained by Oracle.
Enable all audit policiesGenerates statements to enable all audit policies as currently configured in AUDIT_UNIFIED_ENABLED_POLICIES.
Generate Statements Reports

Top Audit Events

ReportDescription
Events by UsersThis query generates a summary of audit events grouped by database username in the unified audit trail. It counts the total number of events for each user and sorts the results in descending order, showcasing the users with the highest frequency of audit events.
Top Events by ActionDisplays a ranking of Action based on the frequency of their associated events in the unified audit trail, offering insights into the most audited databases. Comparable to the script saua_teact.sql.
Top Events by Application ContextDisplays a ranking of Application Context based on the frequency of their associated events in the unified audit trail, offering insights into the most audited databases.
Top Events by Audit TypeDisplays a ranking of Audit Type based on the frequency of their associated events in the unified audit trail, offering insights into the most audited databases.
Top Events by ClientDisplays a ranking of Client Program based on the frequency of their associated events in the unified audit trail, offering insights into the most audited databases. Comparable to the script saua_tehost.sql.
Top Events by Client ProgramDisplays a ranking of Client Program based on the frequency of their associated events in the unified audit trail, offering insights into the most audited databases. Comparable to the script saua_tecli.sql.
Top Events by DBIDDisplays a ranking of database IDs (DBIDs) based on the frequency of their associated events in the unified audit trail, offering insights into the most audited databases. Comparable to the script saua_tedbid.sql.
Top Events by External User IDPresents a ranking of External User IDs by the frequency of their associated events in the unified audit trail, providing insights into the most audited user identities.
Top Events by Global User IDPresents a ranking of Global User IDs by the frequency of their associated events in the unified audit trail, providing insights into the most audited user identities.
Top Events by none Oracle Object NameShows a ranking of non-Oracle object names based on the frequency of associated audit events in the unified audit trail, highlighting the most audited non-Oracle objects. Comparable to the script saua_teobj.sql.
Top Events by Object NameDisplays a ranking of object names based on the frequency of associated events in the unified audit trail, offering insights into the most audited objects. Similar to the script saua_teobj.sql.
Top Events by Object SchemaShows a ranking of Object Schemas by event frequency, emphasizing the policies most often triggered in the unified audit trail. Comparable to the script saua_teown.sql.
Top Events by OS UserDisplays a ranking of OS User based on the number of associated events, highlighting the most frequently triggered policies in the unified audit trail. Comparable to the script saua_teosusr.sql.
Top Events by policesDisplays a ranking of audit policies based on the number of associated events, highlighting the most frequently triggered policies in the unified audit trail. Similar in function to the script saua_tepol.sql.
Top Events by SQL TextSummarizes and ranks SQL statements from the unified audit trail by their event frequency, highlighting the most frequently executed queries.
Top Audit Event Reports

Add Custom Reports in SQL Developer

The SQL Developer already offers many predefined reports. Unfortunately not many in the database security area. However, it is possible to extend the predefined reports with your own reports. See also SQL Developer Reports. The Unified Audit Assessment Reports presented here are bundled in an XML file and available via xx. This XML file can be added relatively easily in SQL Developer. You have the choice to do this as a User Defined Report or Shared Report.

User Defined Reports

Adding user defined reports is relatively simple. Just select the context menue in User Defined Reports and click Open Reports… and select your XML file.

User-defined reports can be modified, renamed, and saved after changes. However, updating these reports directly from an XML file is not straightforward. To incorporate updates from an XML file, the reports must first be removed and then reloaded.

Database / Shared Reports

Database respectivily Shared Reports have a distinct setup process. They must be added through preferences, after which they appear under the Shared Reports tree. Unlike other reports, these cannot be modified directly in SQL Developer, as it only loads the XML file during startup. However, if the XML file is updated — for instance, by downloading a new version — the reports will be updated upon the next restart. For further customization, these reports can still be copied to User Defined Reports for individual modifications.

To add a Shared Report, follow these steps:

  1. Open Preferences in SQL Developer
  2. Navigate to User Defined Extensions
  3. Click Add Row and configure the following:
    • Set Type to REPORT
    • Provide the location path to your XML file.

The screenshot below illustrates an example of this process

SQL Developer Preferences – User Defined Extenstions

Conclusion

The SQL Developer reports presented here, as well as the SQL scripts, are ideal tools for evaluating local audit data quickly and easily. Especially when it comes to getting a first insight into the collected information or when developing new audit policies. For a company-wide overview in the area of database security and auditing, however, there is no way around a central solution. This includes the following solution approaches:

  • Oracle Data Safe A cloud-based security service that automates sensitive data discovery, data masking, database security assessments, and user risk assessments. It’s designed to secure Oracle databases and enhance overall data protection.
  • Oracle Audit Vault and Database Firewall Provides a comprehensive security solution that includes monitoring, analysis, and blocking of unauthorized database activities. It consolidates audit data from multiple sources and offers real-time alerts and reports.
  • Custome Solution Involves developing a bespoke Data Warehouse (DWH) for audit data or utilizing powerful tools like Splunk or Elasticsearch. This approach allows for tailored data aggregation, reporting, and analysis, meeting specific organizational needs. Splunk and Elasticsearch offer advanced data indexing, search capabilities, and visualization tools, making them ideal for handling complex audit data.

I intend to go into more detail about centralized audit solutions in future posts. In the meantime, I hope you can use my scripts to analyze local audit data.

Additional Resources

Some links and references related to this topic.

SQL Developer 19.1 unable to use connection type ldap with OUD

Due to a tip from a work colleague, I came across a changed behaviour of the latest SQL Developer release. It affects the connection type LDAP respectively the use of an LDAP directory for the database name resolution. After specifying one or more LDAP servers it should actually be possible to select the corresponding context in the drop down list. But as of SQL Developer 19.1 this drop down list remains empty as you can see in the following images. As far as I can tell, this problem only occurs when using Oracle Unified Directory and the latest release of SQL Developer.

You can enable debugging in the SQL Developer to search for errors. But it is far easier to have a short look in the OUD access log. In the access log, you will instantly notice an error as you can see in the following excerpt of the access log.

[02/Jul/2019:04:34:51 +0000] CONNECT conn=276 from=172.20.0.1:55224 to=172.20.0.2:1389 protocol=LDAP
[02/Jul/2019:04:34:51 +0000] BIND REQ conn=276 op=0 msgID=1 type=SIMPLE dn="" version=3
[02/Jul/2019:04:34:51 +0000] BIND RES conn=276 op=0 msgID=1 result=0 authDN="" etime=0
[02/Jul/2019:04:34:51 +0000] SEARCH REQ conn=276 op=1 msgID=2 base="" scope=sub filter="(objectClass=orclContext)" attrs="cn,DN"
[02/Jul/2019:04:34:51 +0000] SEARCH RES conn=276 op=1 msgID=2 result=50 message="The request control with Object Identifier (OID) "1.2.840.113556.1.4.319" cannot be used due to insufficient access rights" nentries=0 etime=3
[02/Jul/2019:04:34:51 +0000] SEARCH REQ conn=276 op=2 msgID=3 base="" scope=one filter="(objectClass=orclContext)" attrs="cn,DN"
[02/Jul/2019:04:34:51 +0000] SEARCH RES conn=276 op=2 msgID=3 result=50 message="The request control with Object Identifier (OID) "1.2.840.113556.1.4.319" cannot be used due to insufficient access rights" nentries=0 etime=1
[02/Jul/2019:04:34:51 +0000] SEARCH REQ conn=276 op=3 msgID=4 base="" scope=base filter="(objectClass=orclContext)" attrs="cn,DN"
[02/Jul/2019:04:34:51 +0000] SEARCH RES conn=276 op=3 msgID=4 result=50 message="The request control with Object Identifier (OID) "1.2.840.113556.1.4.319" cannot be used due to insufficient access rights" nentries=0 etime=1
[02/Jul/2019:04:34:51 +0000] UNBIND REQ conn=276 op=4 msgID=5
[02/Jul/2019:04:34:51 +0000] DISCONNECT conn=276 reason="Client Disconnect"

There is an issue with an LDAP control as stated in error message: The request control with Object Identifier (OID) “1.2.840.113556.1.4.319” cannot be used due to insufficient access rights The control 1.2.840.113556.1.4.319 better known as Paged Results Control
allows the client to control the search results. In particular it is used to iterate through the search results a page at a time. And why is this of interest in connection with the SQL Developer?

When using the LDAP connection type, the SQL Developer uses an anonymous LDAP query to query the Oracle context and database name. Starting with SQL Developer 19.1, Oracle seems to use Paged Results Control, which makes sense if you have a large number of LDAP entries. By default Oracle Unified Directory does not allow the Paged Results Control for anonymous connections. Simplest solution to this problem is to allow Paged Results Control also for anonymous connections. For this you have to change the access control handler and the global act either by dsconfig or via OUDSM. The MOS Note 1932191.1 does provide the step by step solution how to modify the global act via dsconfig, OUDSM or by modifying the config.ldif. The latter is not recommended, because this requires a restart of the OUD server. Furthermore, the direct modification of config.ldif is only necessary in a few cases, e.g. when resetting the root user respectively directory manager password.

Procedure to modify the global aci via dsconfig:

  • Start dsconfig and directly modify the act of the Access Control Handler
dsconfig -h oud -p 4444 -D "cn=Directory Manager" -j $PWD_FILE -X set-access-control-handler-prop
  • Select the property global-aci
  • Select remove one or more values
  • Select the # that has Authenticated users control access which contains the 1.2.840.113556.1.4.319 OID)
  • Select use these values
  • Select finish and apply any changes to the Dsee Compat Access Control Handler
  • Start again dsconfig to add the new aci for the Access Control Handler
dsconfig -h oud -p 4444 -D "cn=Directory Manager" -j $PWD_FILE -X set-access-control-handler-prop
  • Select the property global-aci
  • Select add one or more values
  • When prompted with Enter another value for the global-aci property [continue]:, enter
    (targetcontrol="1.3.6.1.1.12|| 1.3.6.1.1.13.1 ||1.3.6.1.1.13.2 ||
    1.2.840.113556.1.4.319 ||1.2.826.0.1.3344810.2.3 ||
    2.16.840.1.113730.3.4.18 ||2.16.840.1.113730.3.4.9 ||
    1.2.840.113556.1.4.473 ||1.3.6.1.4.1.42.2.27.9.5.9")
    (version 3.0; acl "Authenticated users control access"; allow(read) userdn="ldap:///anyone";)
    
  • Select use these values
  • Select finish and apply any changes to the Dsee Compat Access Control Handler

The entire adjustment can also be done directly with two dsconfig commands.

First remove the global-aci for userdn all: (Make sure to remove the line wraps when copy/paste the aci)

dsconfig -h oud -p 4444 -D "cn=Directory Manager" -j $PWD_FILE -X -n set-access-control-handler-prop \
--remove global-aci:"(targetcontrol=\"1.3.6.1.1.12 || 1.3.6.1.1.13.1 || 1.3.6.1.1.13.2 || 1.2.840.113556.1.4.319 || 1.2.826.0.1.3344810.2.3 || 2.16.840.1.113730.3.4.18 || 2.16.840.1.113730.3.4.9 || 1.2.840.113556.1.4.473 || 1.3.6.1.4.1.42.2.27.9.5.9\") (version 3.0; acl \"Authenticated users control access\"; allow(read) userdn=\"ldap:///all\";)"

Add the new global-aci for userdn anyone:

dsconfig -h oud -p 4444 -D "cn=Directory Manager" -j $PWD_FILE -X -n set-access-control-handler-prop \
--add global-aci:"(targetcontrol=\"1.3.6.1.1.12|| 1.3.6.1.1.13.1 ||1.3.6.1.1.13.2 ||1.2.840.113556.1.4.319 ||1.2.826.0.1.3344810.2.3 ||2.16.840.1.113730.3.4.18 ||2.16.840.1.113730.3.4.9 ||1.2.840.113556.1.4.473 ||1.3.6.1.4.1.42.2.27.9.5.9\") (version 3.0; acl \"Authenticated users control access\"; allow(read) userdn=\"ldap:///anyone\";)"

Now the SQL Developer can load the LDAP context and database name.

The question now is whether this change is a bug or a feature. In any case, it would make sense if the SQL Developer could alternatively allow to enter the Oracle context directly and thus avoid the anonymous query.

Some links related to this blog post: