Description
The audit team found that there are effective and efficient regulatory processes, systems and procedures in support of achieving the Canadian Radio-television and Telecommunications Commission (CRTC) statutory mandate.
OPERATIONAL AUDIT REPORT
Findings, Recommendations and
Management Responses
May 2010
Catalogue No. BC92-74/2010E-PDF
ISBN #978-1-100-16159-4
TABLE OF CONTENTS
PREFACE 3
EXECUTIVE SUMMARY 4
INTRODUCTION
Background 6
Scope 6
Audit Objectives 7
Audit Methodology 8
FINDINGS, RECOMMENDATIONS AND MANAGEMENT RESPONSES
Regulatory Processes and Practices 9
Information Systems 13
Monitoring Activities 23
Interface with the Stakeholders 27
Interface between the Sectors 32
Management Control Framework 35
Annex 1 Risk Assessment Framework 49
Annex 2 List of Positions Interviewed 50
Annex 3 Audit Objectives and Audit Criteria 52
Annex 4 Summary: Recommendations
and Management Responses 56
CRTC Operational Audit 2
PREFACE
The operational audit of the CRTC is an initiative of its Chairman, the Honourable
Konrad W. von Finckenstein, Q.C. Shortly after his appointment, the Chairman requested
that an internal audit be undertaken in order to determine the efficiency and effectiveness
of the CRTC’s operations. It was his desire to identify which processes and systems are
working well, and which areas of the CRTC’s operations stood to benefit from increased
attention.
Funding for the operational audit was secured following the roundtable consultations
with CRTC fee stakeholders in 2007. These stakeholder consultations led to a two-year
temporary increase in the CRTC’s spending authority, which allowed the Commission to
undertake a review of its internal processes by outside audit professionals.
Professional audit services were contracted through the Public Works and Government
Services Canada Professional Audit Support Services (PASS) supply arrangement.
Independent oversight of the audit process was provided by the Office of the Comptroller
General’s Small Departments and Agencies Audit Committee (SDAAC). Pursuant to the
recommendation of the SDAAC, and in accordance with the Treasury Board of Canada
Secretariat’s Policy on Internal Audit, the following audit report was approved by the
CRTC Chairman on 25 May 2010.
CRTC Operational Audit 3
EXECUTIVE SUMMARY
Statement of Assurance
This operational audit engagement was planned and conducted in accordance with the
Treasury Board's Policy on Internal Audit and the Institute of Internal Auditors Standards
for the Professional Practice of Internal Auditing.
Audit Opinion
The audit team found that there are effective and efficient regulatory processes, systems
and procedures in support of achieving the Canadian Radio-television and
Telecommunications Commission (CRTC) statutory mandate. In addition, CRTC
initiatives being conducted to enhance regulatory processes, service delivery and
promoting management excellence were found to be effective. The audit evidence
gathered is sufficient to provide senior management with reasonable assurance of the
results derived from this audit.
Summary of Audit Findings
The audit team found that the regulatory processes, systems and procedures were
effective and efficient; that the services were delivered in a fair, open, transparent and
timely basis and in a manner that minimizes the regulatory burden on all stakeholders;
and that the staff was dedicated and prepared to do the work required, meeting the needs
of both industry and the public.
Through participation in the Future Direction initiative, CRTC staff have, in addition to
their regular duties, developed and implemented proposals that will enhance the
regulatory processes, service delivery and promote management excellence.
The existing information systems used to collect and provide information in support of
decision-making and accountability were found to be effective and meeting the needs of
the users. The CRTC has developed a strategic plan that identifies actions to be taken to
upgrade and improve the information systems to increase efficiency and reliability. The
plan has been initiated and is addressing a number of key Information Technology (IT)
priority areas.
Since the CRTC is a regulatory entity, it requires a variety of monitoring mechanisms,
both internal and external. These mechanisms were found to be effective and the results
were being regularly used for regulatory policy and management decision making.
Ongoing and planned initiatives to enhance stakeholder interaction with the CRTC were
found to be effective. A number of projects have been initiated to improve the
distribution and collection of information to and from stakeholders.
CRTC Operational Audit 4
The interface between the four sectors of Broadcasting, Telecommunications, Policy
Development and Research, and Corporate Services and Operations was found to be
effective in supporting the enhanced regulatory processes and results. An appropriate
governance structure was in place that ensured management direction, and plans and
actions were understood, appropriate and responsible.
The management control framework in place provides a clearly defined strategic
direction that is aligned to the CRTC mandate. The 3-Year Work Plan, the Report on
Plans and Priorities, and the Program Activity Architecture address the organization’s
outcomes, priorities and related action plans. A Performance Measurement Framework
was recently enhanced and will improve the ability of the CRTC to report performance in
future Departmental Performance Reports.
Summary of Audit Observations
The audit team identified four primary management areas that it believes, when
addressed, will enhance processes, systems and practices.
The project management authority for the key area of streamlined rules and regulations as
well as other initiative activities should be clarified and progress reporting should be
complete and timely.
An integrated planning and reporting process should be developed and implemented, and
a pre-set forward agenda established to ensure that all major planning and reporting
elements are reviewed at specified times throughout the year.
The responsibility for financial monitoring should be reiterated to Responsibility Center
Managers.
The Five Year IM/IT Roadmap should identify the specific projects for each of the five
years, the estimated project cost and how each project ties into the IT and corporate
strategic objectives.
Further elaboration on the above four management areas, as well as administrative areas
of potential improvement, are provided in the body of the audit report.
Management Action
The CRTC is addressing the recommendations in this report. Specific responses to each
recommendation are provided in the body of the audit report.
CRTC Operational Audit 5
INTRODUCTION
Background
The Canadian Radio-television and Telecommunications Commission (CRTC) was
established by Parliament in 1968. It is an independent public authority constituted under
the Canadian Radio-television and Telecommunications Commission Act and reports to
Parliament through the Minister of Canadian Heritage.
The CRTC is vested with the authority to regulate and supervise all aspects of the
Canadian broadcasting system, as well as to regulate telecommunications common
carriers and service providers that fall under federal jurisdiction. The CRTC derives its
regulatory authority over broadcasting from the Broadcasting Act. Its
telecommunications regulatory powers are derived from the Telecommunications Act and
the Bell Canada Act.
The CRTC serves the public interest by maintaining a balance between the cultural,
social and economic objectives of federal legislation on broadcasting and
telecommunications, taking into account the wants and needs of Canadian citizens,
industries and interest groups.
The CRTC fulfils its regulatory responsibilities by means of a number of inter-related
tasks, including:
• issuing, renewing and amending licences for broadcasting undertakings;
• holding public hearings on matters of significant public interest;
• making determinations on mergers, acquisitions and changes of ownership
in the broadcasting industry;
• approving tariffs and agreements for the telecommunications industry;
• fostering increased reliance on market forces for the provision of
telecommunications services and ensuring that regulation, where required,
is efficient and effective;
• monitoring competition and removing obstacles to competition;
• collaborating with industry to resolve competitive disputes;
• developing and implementing regulatory policies with a view to meeting
the objectives of the Broadcasting Act and the Telecommunications Act;
• monitoring, assessing and reviewing, where appropriate, regulatory
frameworks to meet policy objectives; and,
• monitoring the programming and financial obligations of broadcasting
undertakings to ensure compliance with regulations and conditions of
licence.
Scope
The operational audit included the review of the CRTC planning, regulatory and
reporting processes in the four sectors of Broadcasting, Telecommunications, Policy
CRTC Operational Audit 6
Development and Research and relevant sections within the Corporate Services and
Operations sector (i.e. Planning and Processes). It also included a review and assessment
of the on-going and planned initiatives to enhance regulatory processes, service delivery
and promote management excellence.
The CRTC budget allocation for fiscal 2009/10 is approximately 425 FTEs and a planned
spending of $46.0 M.
The audit was conducted at the CRTC headquarters in Gatineau, Quebec, during the
period April to September 2009. The audit focussed on the period 2008 to the present.
Audit Objectives
The overall objective was to review and assess: a) the effectiveness and efficiency of the
existing regulatory processes, systems and procedures in support of achieving the
CRTC’s statutory mandate; and, b) the CRTC initiatives (completed, on-going and
planned) being conducted to enhance regulatory processes, service delivery and promote
management excellence.
Specifically, the audit focused on the following six objectives:
1. The effectiveness and efficiency of regulatory processes and practices (e.g. public
hearing processes, application processes, scheduling, etc.) and to ensure that the
services are delivered in a fair, open, transparent and timely basis and in a manner
that minimizes the regulatory burden on all stakeholders. This effort included an
assessment of the on-going and planned CRTC initiatives to enhance the
regulatory processes and service delivery.
2. The effectiveness of existing information systems that are used to collect and
provide information in support of the CRTC regulatory processes (e.g. the
reliability of information systems for decision-making and accountability
purposes).
3. The effectiveness of monitoring activities (including performance monitoring
reports) and the extent to which the results of monitoring activities are used for
regulatory policy and management decision making.
4. An assessment of the effectiveness of on-going and planned initiatives that
address: a) the adequacy of information provided to regulatory stakeholders on
the CRTC website (i.e. to inform them of CRTC plans, process, service delivery
standards, documentation requirements and results achievement); b) the CRTC
actions taken to address stakeholders’ comments received as part of industry
surveys (e.g. dealing with issues such as stakeholder satisfaction and
recommendations for process improvement); and, c) the current processes of
consultation with the CRTC’s stakeholders in the setting and adjusting of the
CRTC Operational Audit 7
CRTC’s annual plans (e.g. is the CRTC asking its stakeholders the right questions
and is it capturing the right information?).
5. An assessment of the effectiveness of interfaces by the four sectors, with other
sectors of the CRTC (e.g. legal, strategic communications and commission
members) in support of enhanced regulatory processes and results.
6. The effectiveness of the CRTC management framework (e.g. practices and
procedures relating to planning, organizing, controlling, leading, communications
and management of human and financial resources) within each of the sectors
noted above.
Audit criteria were developed to assess each of these six objectives.
Audit Methodology
The audit was conducted in accordance with generally accepted auditing standards and
the requirements set out in the Treasury Board’s policy on Internal Audit. The audit
methodology respected the Office of the Auditor General’s methodology for Performance
Audits (formerly known as value-for money audits). The key tasks associated with the
audit approach involved reviewing CRTC documentation, systems and processes, and
interviewing CRTC staff.
To obtain an appreciation of the organization and its environment, management concerns
and insight into CRTC governance, risk management and controls, the audit team
completed a preliminary survey that included reviewing a number of documents and
interviewing members of the management team. The results of this review were used to
develop and discuss with management the audit program to use during the examination
phase of the audit. The Audit Team followed the submitted audit program.
CRTC Operational Audit 8
FINDINGS, RECOMMENDATIONS AND MANAGEMENT RESPONSES
Regulatory Processes and Practices
Conclusion
The audit team concluded that the CRTC has effective and efficient regulatory processes
and practices in place to ensure that the services are delivered in a fair, open, transparent
and timely basis as evidenced by our review of both the Planning and Process Directorate
procedures and the compilation of the briefing documentation prepared for the
Commission briefing books. Both processes ensured all submitted public comments and
interventions are considered during the decision-making process and the Commission
briefing books demonstrated that they contained a comprehensive summary of the
information required for decision-making. In addition, the audit team concluded that the
CRTC is actively pursuing initiatives that minimize the regulatory burden on
stakeholders and is addressing the profound change in the communications industry as a
result of technological and corporate convergence. This was demonstrated by our review
of the CRTC management framework over the many initiatives underway or completed
to address both the government’s policy direction to encourage increased reliance on
market forces in the telecommunication industry and the changing digital technologies
resulting in a convergence of the traditional industries.
Public hearing, scheduling and application processes
The Planning and Process Directorate oversees all public hearings. It is responsible for
completing tasks with respect to the scheduling, administration and logistics associated
with the hearing process. The unit is responsible for ensuring that all applications are
placed on the CRTC website for public awareness and comment, issuing the public notice
through the appropriate media outlets, forwarding the application to the appropriate
designated functional analyst and ensuring that interested parties have an opportunity to
file comments, interventions and replies. All negative interventions, counter
interventions and a summary of positive comments are assembled by the Directorate and
included in the briefing books. The audit team concludes that the Directorate is
following all applicable procedures, thereby ensuring all public comments and
interventions are considered during the decision-making process. This conclusion was
based on the interviews completed with the responsible managers, documentation review
of selected briefing books and review of the publicly accessible files on the CRTC
website.
The review of the briefing documentation prepared for the Commission briefing books
established that they contained a comprehensive summary of the information required for
decision making. In addition, interviews with responsible managers confirmed that the
Commissioners were satisfied with the level of support being provided. Our review of
the files indicated that service standards were consistently met resulting in no occurrences
of work not completed on time. Interviews with responsible managers established that
CRTC Operational Audit 9
CRTC Operational Audit 10
uncompleted work was an unacceptable option and that the staff would do whatever was
required to meet the deadline.
Increased workload has been generated by a significant increase in the number of public
hearings that are held each year
1
. In addition to a desire for greater transparency and
openness, this increase has been caused by certain administrative review processes now
being subject to the demands of a public hearing; a number of policy issues that have
become due for review; the change which is being experienced within the industry;
increased demand for new radio licenses; and the current state of the economy, which has
resulted in an increased frequency of ownership change.
The problem most frequently expressed by staff was the workload swings that were
necessitated by the scheduling of hearings. While there was some concern expressed at
the Executive Director level regarding the potential for over tasking of staff and staff
burn out, the majority of the managers interviewed expressed the opinion that they had
adequate resources to complete the assigned tasks. Research and policy analysis are not
the type of activities that would benefit from traditional work measurement assessment.
Determination of whether an adequate level of resourcing had been achieved must be
addressed by questions such as whether the work is being completed in an acceptable
manner within the required time frames and whether the associated resources appear to
be either underemployed or over worked.
The review of related documentation and discussions with staff revealed that:
• There is adequate subject-matter expertise for the conduct of policy research with
the majority being completed by CRTC staff with contracting out being utilized
when there is a requirement for specific expertise or third party advice;
• A process exists to solicit stakeholder comments related to service delivery and
significant change initiatives and management actions are communicated to the
appropriate stakeholders on a timely basis;
• CRTC issues a public notice that describes a proposed change and requests that
comments be received by a specific date when developing or changing a policy;
• Proceedings attract participation from opposing interests and public interest
groups and the degree of participation will vary depending on the subject matter.
The CRTC has established an administrative process that adequately addresses
and supports this participation;
• Established service standards were met as demonstrated by the file review;
• All comments and reply comments received from all stakeholders are made a part
of the public record of proceeding with material being treated as confidential and
excluded in limited circumstances when public disclosure could create
competitive disadvantage; and,
• All decisions and associated rationale are posted on the CRTC website.
1
Public Hearing statistics maintained by the Planning and Process Directorate indicated the following:
2005/06 – 13 hearings, 27 days;
2006/07 – 16 hearings, 46 days;
2007/08 – 20 hearings, 60 days; and,
2008/09 – 18 hearings, 66 days.
Initiatives to enhance the regulatory processes and service delivery
In recent years there have been two significant influences that have required theCRTC to
review the delivery of its mandate. The government issued a policy direction in
December 2006 on thederegulation of the Canadian telecommunication industry and is
placing an increased emphasis on market forces. In addition, the communications
industry is undergoing a profound change as new digital technologies transform how
Canadians communicate, how the CRTC delivers its mandate and how the public is
informed and entertained. The broadcasting and telecommunications industries are
embracing new technologies and moving to a convergence of their traditional industries.
The audit team reviewed the management framework that was in place to address the
CRTC response to these influences. In 2008, an internal task force was created to
develop streamlined rules and regulations. The task force developed terms of reference
articulating the objective, scope, background and team membership. The project
manager’s initial action was to identify and document all activities generated by both the
policy direction and the Future Direction initiative. The audit team reviewed the
resulting activity catalogue with the responsible managers and found it to be an effective
management tool, except for concerns raised by the audit team under the Management
Control Framework section of this report.
The CRTC has implemented a number of initiatives to minimize the regulatory burden in
the industry and streamline its regulatory processes. Some examples are as follows:
• Under the realigned structure, activities common to both broadcasting and
telecommunications were undertaken by the Policy, Development and Research
Sector. PDR has been tasked to focus on convergence policy, social policy and
arbitration. These activities will become more important in an increasingly
deregulated industry.
• The CRTC’s regulatory policies were reviewed and provided auditors with
sufficient evidence showing how the CRTC eliminated certain data reporting
requirements.
• In telecommunications, process improvements have been ongoing as a result of
the policy direction and other initiatives implemented during the last few years.
An example is the implementation of streamlined processing (10 day approval) to
assess requests for retail tariff modifications.
• The Broadcasting and Telecommunications Rules of Procedures have been
combined into a single set of rules that incorporates input from CRTC senior staff
and is currently under review at the Department of J ustice. It was expected to be
ready for publication and implementation during fiscal year 2009/10, however
implementation is now forecast for fiscal year 2010/11.
CRTC Operational Audit 11
• The CRTC is currently in the process of reviewing the procedures in the Public
Hearing process and has identified opportunities for process development.
Process maps of the three major public hearing processes were developed which
reveal “hot spots” where process improvements could be made.
• The CRTC has expanded the scope of the exemption order respecting radio and
television temporary network special event type 1 undertakings to permit these
undertakings to distribute programming that is of a special and recurring nature
(i.e. CRTC exemption order 2009-18) resulting in less filing by the industry.
• The CRTC had discussions with stakeholders which resulted in the
implementation of streamlining measures pertaining to the filing of channel line-
ups on renewal, application forms, replacing paper services area maps with maps
in electronic format and eliminating financial information from the application
form for new BDUs (i.e. CRTC Info Bulletin 2009-384).
• Streamlining the process for transactions involving the transfer of shares (i.e.
Information Bulletin Broadcasting Circular CRTC 2008-8).
• Internal processes have been put in place for transactions involving the
acquisition of assets and certain transfers of shares to ensure that a draft decision
is distributed on time to the Panel and the Full Time Members (FTM) for
consultation to allow publication of the decisions within 35 days from the first
day of the public hearing.
• Annual no change confirmation or updating versus a complete package of
ownership information accompanying licence renewal applications.
• All licences under the same ownership being renewed as a group at the same time.
• The CRTC introduced Telecom Regulatory Policy 2009-183, which eliminated
certain data reporting requirements such as reporting on pay-telephone
competition and tracking customer complaints regarding modem hijackings. In
addition, telecommunications service providers are no longer required to file an
annual report on the affordability of telephone services. The CRTC will continue
to gather this information from other sources.
• Broadcasting Circular CRTC 2005-466 allowed applicants to file their
applications and all related documents in electronic form using ePass. Using
ePass eliminates the requirement to file a hard copy of the documents. This
expedited the filing process by removing the need for the CRTC to verify
electronic and hard copy versions of an application.
• An MOU was developed between the CRTC and Statistics Canada concerning the
collection and sharing of information for telecommunications and broadcasting
surveys. This arrangement was intended to ensure that the information released
CRTC Operational Audit 12
by the Parties is as consistent as possible, that the Parties do not duplicate efforts,
and that no undue burden is imposed on respondents.
• The CRTC updated the look and feel of its website in February 2009.
The audit team concluded that theCRTC is actively engaged with its stakeholders to
minimize the regulatory burden in the industry and streamline its regulatory processes.
This was demonstrated by the many initiatives completed and implemented, and the
management framework that theCRTC has in place to manage the initiatives still
underway.
Information Systems
Conclusion
The audit team concluded that the Information Systems used to collect and provide
information in support of the CRTC regulatory processes are effective in providing
reliable information for decision making and accountability. This was demonstrated by
the CRTC website that provides the public and staff access to information and
applications. The use of pre-set electronic forms requiring specific data to be input into
fields minimizes the risk of inaccurate data. In addition, analysts review key information
to further ensure its accuracy. The audit team noted that the CRTC has policies in place
to ensure information is compliant with Privacy Legislation; however, the CRTC has
completed only two privacy assessments. The CRTC should conduct a risk assessment
on all CRTC key databases, and the databases considered to be at risk should be subject
to a Privacy Impact Assessment. Although the CRTC has adequate back up systems to
ensure continuity of information and systems, it does not have an alternate data
processing site. The CRTC should assess whether or not it believes this is an acceptable
risk and if determined to be unacceptable, the CRTC should identify a location for an
alternate data processing site. The CRTC conducted a Threat and Risk Assessment in
2001, which should be updated. The audit team noted that IM/IT strategic planning was
adequate; however, it did not identify the estimated costs of specific projects and how
each project was integrated into the IT strategic objectives and the CRTC 3-Year Work
Plan.
This conclusion is based on the audit team’s review of the CRTC policies related to data
and information. Auditors conducted reviews to provide reasonable assurance that the
CRTC complied with Privacy Legislation. The CRTC website was used by the auditors
to ensure it provided easy access to decisions, information and services. The audit team
observed data entry into certain systems and at the same time observed information
displayed on the screens of these systems to obtain reasonable assurance of information
accuracy. It observed demonstrations of the key functions within the Ownership
Structure of the Industry (OWN), Application Support System (APP), Master Address
Database (MAD), Data Collection System (DCS), Radio Assessment of Industry (RAP)
and RAPIDS applications. IM/IT plans were reviewed to determine whether these plans
CRTC Operational Audit 13
were linked to the CRTC business plans and IM/IT business cases provided some
information concerning changes to systems.
User service and access
The CRTC website provides easy access to decisions, information and services as well as
the submission of applications, interventions and comments. The website, combined with
the CRTC Intranet site (“the Zone”), provides both the public and internal CRTC users
access to information and applications. For example, the Ownership system allows
analysts to access the system for both entering data and making inquiries. Personnel in
other areas have access for inquiries, using a more extensive inquiry facility. An
interface with the APP system provides OWN with updated identification information on
a periodic basis.
Accuracy of transaction coding and processing
Input of transactions to the various components of the applications is accomplished
through pre-set electronic forms, which require specific data to be input into fields. Since
the information is provided electronically, it is very difficult to obtain the original direct
source data as evidence to ensure data accuracy. However, the procedures observed
during the audit provided assurance that the data is accurate. For instance, analysts
review key information (e.g. applications, surveys, forms) to ensure accuracy.
The CRTC introduced its web-based Data Collection System (DCS) in J anuary 2004.
DCS employs a secure, encrypted connection between the entity submitting data and the
CRTC thereby improving the accuracy, ease and timeliness of data submission.
External clients using ePass expedite the data filing process by removing the need for the
CRTC to verify both electronic and hard copy versions of an Application. Data
submitted directly by external clients without being input by CRTC staff is as accurate as
possible. The CRTC informed the audit team that some stakeholders had experienced
difficulty when submitting very large application files using ePass, since PWGSC has
restricted and reduced the size of the files being downloaded. The CRTC has reviewed
the comments and problems reported by stakeholders and will be improving the ePass
process through the implementation of a new software security solution. This will result
in a more effective and efficient electronic application system to meet stakeholder
demands. In the meantime, the stakeholders can utilize the CRTC central point of contact
to address their inquiries.
Records/information maintained in accordance with laws and regulations
The CRTC has several policies related to records, data and information that provide
assurance that the records, data and information are in compliance with Privacy
Legislation. These policies include the CRTC Policy on the Identification and Release of
Information, Guidelines on the Release and Protection of Information, Guide to
Information Protection and Destruction, and the CRTC Directive on Email Management.
CRTC Operational Audit 14
All databases are secured using a user-id/password and managers must approve each
employee’s access to applications using each database. User profiles restrict access to
read, write or update information/data on the database.
Multiple firewalls are in place. Intrusion detection and protection are set up to monitor
both internal and external traffic.
The CRTC subscribes to a software service that allows the CRTC to remotely wipe out a
laptop hard disk of any web-connected laptop reported to be missing.
The CRTC is responsible for demonstrating that its collection, use and disclosure of
personal information respect the Privacy Act and privacy principles throughout the
initiation, analysis, design, development, implementation and post-implementation
review phases of their program and service delivery activities. This is accomplished
through the Privacy Impact Assessment (PIA) Policy.
The PIA Policy is one of several tools designed to assist Canadians in understanding how
the government handles their personal information and to trust it to do so responsibly.
The policy is based on privacy principles common to all data protection regimes. These
principles are enumerated in the "Code of Fair Information Practices" in the federal
Privacy Act as well as in the ten privacy principles attached to the Personal Information
Protection and Electronic Documents Act.
The PIA Policy states that Institutions must identify all personal information associated
with business processes. This should be accomplished by developing a detailed
description and analysis of the data flows. Key components include the business process
diagrams, data flow tables and system and infrastructure architectures. The Policy states
that departments and agencies must ensure and document that privacy principles,
legislation and policies are adhered to. The CRTC does this through its Important
Notices on the website.
The PIA Policy does state that institutions must document their evaluation of privacy
risks, the implications of those risks and their discussions of possible remedies, options
and recommendations to avoid or mitigate such risks.
The end result of the PIA Policy is assurance that all privacy issues have been identified
and resolved or mitigated. The associated documentation forms the basis for seeking the
advice of and notifying the Privacy Commissioner as well as for assuring the public that
privacy issues have been addressed.
The CRTC has completed only two PIAs. One was on the Do Not Call List (DNCL) and
a second was on the Data Collection System (DCS). In 2004 a preliminary PIA was
completed on the APP system and was intended to be fully completed prior to proceeding
with the implementation of the ePass initiative. However, the complete PIA for the APP
system has not yet been completed.
CRTC Operational Audit 15
A PIA is required when there is new data matching due to changes in the business
procedures and/or systems. In absence of a PIA completed on other databases, the CRTC
is at risk of not complying with the "Code of Fair Information Practices" in the federal
Privacy Act or the ten privacy principles attached to the Personal Information Protection
and Electronic Documents Act. Without a robust PIA process, the CRTC is at risk of
disclosing personal information without appropriate authorization.
The CRTC has a Privacy Statement on its website, although there have been only two
PIAs completed. The Privacy Statement indicates that the CRTC does not automatically
gather personal information and that personal information is only gathered if it is
supplied by clients and/or the public. The Privacy Statement also states that “The CRTC
will collect, use and disclose personal information in accordance with the Privacy Act”.
In order to demonstrate compliance with the above statement the CRTC requires a more
robust PIA process that ensures all its databases are at least assessed as to the risk that
personal information could be provided in a way that is not compliant with the Privacy
Act. The audit team did not find any examples of information being disclosed
inappropriately.
Recommendation:
The Director of IM/IT should conduct a risk assessment on all CRTC key
databases and the databases considered to be at risk should be subject to a
Privacy Impact Assessment.
Risk Type Audit Risk Rating Impact
Compliance Moderate Potential that personal information could
be provided in a way that is not compliant
with the Privacy Act. Personal
information could be disclosed.
Management Response:
Response Office of
Primary
Interest
Timeline
A consultant will be hired to conduct a Threat and
Risk Assessment (TRA) on the DCS system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a Privacy
Impact Analysis (PIA) on the OWN system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a TRA on
the CRTC network.
Director
IM/IT
Summer
2010
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be carried
out on those priority systems revealed to be at
Director
IM/IT
Summer
2010
CRTC Operational Audit 16
significant risk.
All remaining databases will undergo PTRAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary PIAs
(PPIA). Full PIAs will subsequently be carried out
on those priority systems revealed to be at
significant risk.
Director
IM/IT
October
2010
All remaining databases will undergo PPIAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
Continuity of information and systems
Databases and Data Stores are backed up at regular intervals and backup media are sent
to the National Archives.
The Treasury Board Secretariat in the TBS Business Continuity Plan (BCP) standard
identifies four elements of a business continuity program as:
• The establishment of a BCP governance structure;
• The conduct of a business impact analysis;
• The development of business continuity plans and arrangements; and,
• The maintenance of BCP program readiness.
The audit team reviewed the CRTC Business Continuity Plan Program Guide and found
that the TB Standards had been met, with the exception of training and testing of the plan.
The audit team found no evidence that the training had been provided or that the Business
Continuity Plan had been tested. The audit team also noted that there is no data centre
alternate processing site in the Continuity Plan. Without an alternate processing site, the
CRTC is at risk of not being able to provide service to its clients and the public. Clients
would not be able to submit data, hearing dates would be unavailable and notices would
not be published. Although there is no government standard regarding an alternate data
center processing site, it is considered a best practice. CRTC management should be
aware of all the risks associated with not having an alternate data center processing site.
CRTC Operational Audit 17
Recommendation:
The Director of IM/IT, in collaboration with senior management, should:
• Conduct a risk assessment to determine whether or not the CRTC is
at risk by not having an alternate data center processing site. If the
risk is deemed unacceptable, the CRTC should identify an
appropriate location to set up an alternate data processing site; and,
• Test the Business Continuity Plan and document the results.
Risk Type Audit Risk Rating Impact
Operations Minor Potential that CRTC could not provide
service to its clients and the public.
Management Response:
Response Office of
Primary
Interest
Timeline
IM/IT staff will test the current Business
Continuity Plan (BCP) to determine whether it
meets legislative and operational requirements.
Results and associated recommendations will be
presented to the Operations Committee.
Director,
IM/IT
Fall 2010
Data center monitoring
The audit team noted that there were no monitoring reports on the performance of the
CRTC data center. Typical monitoring and reporting activities would include the amount
of “up time”, CPU utilization, disk utilization and network utilization. The CRTC is at
risk of not being able to identify, at an early stage, potential processing problems. A
rigorous monitoring and reporting regime on the performance of the data center would
mitigate the risk.
Recommendation:
The Director of IM/IT should implement a rigorous monitoring and reporting
regime on the performance of the data centre.
Risk Type Audit Risk Rating Impact
Monitoring Moderate Potential process problems are not
identified for remedial action at an early
stage.
CRTC Operational Audit 18
Management Response:
Response Office of
Primary
Interest
Timeline
A monitoring system called System Center
Operations Manager (SCOM) has been
implemented to record service information
related to the data center.
Director, IM/IT Spring 2009
Best practices for monitoring and reporting
regimes are currently being reviewed.
Director, IM/IT Spring 2010
Appropriate monthly monitoring and reporting
system to be implemented as required.
Director, IM/IT Fall 2010
Threat and risk assessment
The last Threat and Risk Assessment (TRA) was conducted in March 2001 where the
threats and risks were deemed Medium to Low. Since the TRA aids in the determination
of security requirements, the CRTC should conduct a TRA for every program, system or
service. For example, a TRA should be conducted for physical security of the data
centre; a TRA on Spyware/Viruses; a TRA on the electronic transmission of data; a TRA
on the Application Development Process on access controls; and, a TRA on some of the
larger systems such as APPS, DCS, Finance, electronic hearings, etc. The TRA will
assist the CRTC to answer the following key questions:
• What needs to be protected?
• Who/What are the threats and vulnerabilities?
• What are the implications if data were damaged or lost?
• What is the value of data to the organization?
• What can be done to minimize exposure to data loss or damage?
Threat and Risk Assessments can be short and simple or detailed and rigorous, depending
on the sensitivity, criticality and complexity of the program, system or service being
assessed.
Recommendation:
The Director of IM/IT should update the CRTC list of risks by conducting a
Threat and Risk Assessment of all services.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential security threats go undetected.
CRTC Operational Audit 19
Management Response:
Response Office of Primary
Interest
Timeline
Director IM/IT The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be
carried out on those priority systems revealed
to be at significant risk.
Summer
2010
Director IM/IT All remaining databases will undergo PTRAs
as they are modified according to the existing
project life cycle. Business cases presented to
the IM/IT Steering Committee during the
project proposal cycle will include explicit
reference to the importance of carrying out
these assessments.
December
2010 and
ongoing
Strategic planning for information systems
The IM/IT Roadmap presented in J une 2009 is an example of a good planning document.
It identifies the actions to be taken to upgrade the IM/IT environment for increased
efficiency, reliability, mobility and around-the-clock use from CRTC workers. It
provides a five year course of action based on principles generally found in the CRTC 3-
Year Work Plan such as “establishing modern and secure infrastructure to meet
stakeholder demands”.
Some of the projects discussed in the roadmap include:
• Revamp the CRTC web presence and migrate to a Web 2.0 environment;
• Develop an approach for CRTC content management and web publishing;
• Review the current tools, processes and procedures used in exchanging
information between the CRTC, the industry and the public; and,
• Develop a Business IM/IT Partnership Charter spelling out the principles,
mechanisms and approaches to be used in establishing and operating the
partnership.
The five year IM/IT Roadmap, as discussed with the IM/IT Steering Committee in March
2009, was developed using a discussion guide that described the scope of the roadmap as
a way for the CRTC to move forward with respect to:
• Enhancing the alignment between its business lines and IM and IT;
• Enabling business lines to extract maximum business value from their IM and IT
investments; and,
• Creating appropriate success conditions for alignment and IM and IT business
contribution.
CRTC Operational Audit 20
Although the IM/IT Roadmap was a good starting point, it did not identify the specific
projects for each of the five years, and it did not identify project estimated costs or how
each project ties into the IT Strategic Objectives or the six coordinated actions.
Estimated costs and linkages to the IT strategic objectives or six coordinated actions were
not found in any other document provided to the audit team. All the information should
be in one document facilitating management retrieval.
For example, the IM/IT Proposed Projects for 2009/10 (May 24, 2009) lists the IM/IT
accomplishments for 2008/09 and lists the carry over projects for 2009/10; however, it
does not show how the completed projects or the carry over projects link to the six
coordinated actions of:
• Infrastructure enhancement;
• Rationalization of systems, tools and processes;
• Enhanced stakeholder interfaces;
• Effective Information and Knowledge Management;
• Effective Partnership; and,
• Systemic Organizational Innovation.
The IM/IT Roadmap does not show how the IM/IT projects are linked to the three
Commission priorities found in the 3-Year Work plan for 2008 to 2011 of:
• A more focused regulatory approach;
• Greater outreach to stakeholders; and,
• An improved organization.
Although there is no documentation to show the direct linkage between the IM/IT
Roadmap and Business Cases to the 3-Year Work Plan and RPP, the audit team was able
to link some of the projects. For example:
• The Upgrade to Ownership application is based on the 3-Year Work Plan (year
2008/09) for implementing a broadcasting ownership reporting mechanism; and,
• SQL2005 upgrade, SAN upgrade, Vista Upgrade, and other software
implementation are all based on the 3-Year Work Plan to establish modern and
secure infrastructure to meet stakeholder demands.
An IM/IT Steering Committee was created for approving IM/IT projects and ensuring
that the CRTC has policies, technologies and the infrastructure to conduct business
electronically with all stakeholders. Members of the Committee bring forward priorities,
issues and problems. The Committee consists of members from IM/IT, Broadcasting,
Telecommunications, Media/Policy Development and Research, Communication, Legal
and Finance. It meets three times a year.
CRTC Operational Audit 21
Although this process is used to approve projects, there is no documentation showing
how each approved project ties into the IT Strategic Objectives, the six coordinated
actions or to the 3-Year Work Plan.
Without linking the IM/IT projects to the IM/IT Strategy or to the 3-Year Work Plan,
there is a risk that resources will be used on projects that do not further the objectives of
the CRTC.
Recommendation:
The Director of IM/IT should enhance the IM/IT Roadmap by identifying specific
projects, their estimated costs and how each project ties into the IT strategic
objectives, and the 3-Year Work Plan.
Risk Type Audit Risk Rating Impact
Strategy Minor Potential that resources are used on
projects that do not further the objectives
of the CRTC.
Management Response:
Response Office of
Primary
Interest
Timeline
All suggested information will be incorporated
into the IM/IT Roadmap during the next
project proposal cycle, with projects for the
next 3 years clearly linked to the CRTC
priorities and 3-Year Work Plan through the
use of business case templates.
Director, IM/IT Summer
2010
Project business case
A project business case provides a detailed investment proposal, including an analysis of
the costs, benefits and risks associated with a proposed investment and offers reasonable
alternatives. It provides information necessary to make a decision about whether or not a
project should proceed. It is the indispensable first activity in the lifecycle of an IT
investment.
The audit team reviewed eight project business cases and found that they are incomplete,
do not define accountability and do not adhere to IT Business Case standards.
Specifically, business cases lack the following:
• Benefits of the solution are not quantified;
• There is no project sponsor;
CRTC Operational Audit 22
• There is no project manager;
• There is no discussion of the different options studied to solve the business
problems;
• There are no timing, resources required or cost estimates; and,
• There is minimal risk identification.
Recommendation:
The Director of IM/IT should include in all project business cases the following
elements:
• Summary of alternatives considered;
• Resources required;
• Identification of risks for not approving the project;
• Identification of risks with mitigation strategies for the project;
• Reason(s) for selecting the recommended solution; and,
• Project management accountability framework.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential that not all pertinent information
is assessed before making the investment
decision.
Management Response:
Response Office of
Primary
Interest
Timeline
All future business cases presented to the
IM/IT Steering Committee will adhere to the
existing Business Case standards identified by
the auditors.
Director, IM/IT Summer
2010
Monitoring Activities
Conclusion
The audit team concludes that overall the monitoring activities are effective and are used
for regulatory policy and management decision making. This was demonstrated by the
audit team’s review of the current suite of monitoring reports available to the CRTC staff.
All managers interviewed commented favourably that the systems providing the reports
were constantly being updated as a result of both internal and external requests for system
updates.
This conclusion is based on conducting interviews and observing how certain IT systems
collect and report on information. In addition, the audit team noted the quality assurance
CRTC Operational Audit 23
practices of data submitted by external clients as performed by CRTC analysts. Various
monitoring reports were reviewed along with information obtained from the CRTC
website.
Regulatory monitoring
The Communication Monitoring Report was reviewed to determine what information was
being monitored and reported. The report was also used to identify which IT systems
were used to collect and report data.
There are several systems that collect information used by the CRTC to facilitate the
monitoring of regulatory policy. These systems include:
• The Ownership Structure of the Industry (OWN) - This system maintains
information on the ownership and control structure of companies and individuals
involved in the Canadian Broadcasting industry. It assists the CRTC to determine
the entity’s eligibility to hold a licence, the extent of Canadian control and all
entities involved in the control chain;
• The Application Support System (APP) – This system is used to register
applications filed by the Broadcasting and Telecommunications industries. It
provides the ability to disseminate the application to the processing team,
schedule the process, monitor the progress and report activities and statistics for
all broadcasting and telecommunications applications, decisions, licences and
undertakings. It also provides the calendar of activities for the CRTC, APP
Registers, schedules and tracks reports, and produces statistics for all
Broadcasting and Telecom applications, licences, undertakings, decisions and the
CRTC Calendar of Activities;
• The Radio Assessment of Industry (RAP) – This system is a tool for analysts to
assess logger tapes submitted by radio stations for the purpose of monitoring
compliance to the regulations and to allow for the collection and analysis of
popular music data; and,
• The Data Collection System (DCS) – This system is a web based system that
archives, reports, and analyzes data related to competition in Canadian
telecommunications markets. The system supports the CRTC activities relating to
preparing the annual Report to the Governor in Council (GIC): Status of
Competition in Canadian Telecommunications Markets -
Deployment/Accessibility of Advanced Telecommunications Infrastructure and
Services; produces a Telecommunications registration list membership;
international telecommunications licence renewals; telecommunications fees; and,
the Contribution Collection Mechanism (CCM). The system also contains contact
details of individuals and entities needed to administer and collect the previously
mentioned data, pursuant to Telecom Circulars 2003-1 and 2005-4. These
Circulars describe the annual collection of telecom industry data using DCS and
also describe administrative changes in the annual collection of telecom industry
data that the Commission implemented to better coordinate and streamline a
number of activities it undertakes to regulate and monitor the Canadian
CRTC Operational Audit 24
telecommunications industry. The annual data collection is also used to support
CRTC activities relating to maintaining and updating telecom entity registration
lists, international licences and telecom fees.
Information from these systems is used by the CRTC staff responsible for regulatory
monitoring and policy decision making. These systems are constantly being upgraded to
accommodate requests from internal and external sources in order to make processing of
applications and data more efficient.
Monitoring report review and approval process
While conducting the audit fieldwork, it was noted that reports are reviewed by Analysts
and Managers before they are actually distributed or posted onto the CRTC website. The
CRTC Web Publishing Guide states that all web content changes or additions must be
signed off by the designated content approver (i.e. Director or Director General
depending on content) before they can be submitted for web publishing.
Another example relates to the decision process used to approve the publishing of the list
of radio stations tested for compliance with Canadian content requirements. After the
analyst creates the report from the data in the RAP system, the report is sent to the
manager for final approval before posting the report to the CRTC website.
The Data Collection System (DCS) is used by clients for input of data for related
statistical reporting/collection. The data is input using pre-set electronic forms. After the
data is input by the clients the responsible analysts perform a quality assurance review on
the data. Once the data is deemed to be reasonable, the analysts notify clients to formally
submit the data into the system.
After analyzing radio programming information sent by radio stations, analysts produce a
report called the “Analysis Report”. The report is approved by management prior to
distribution.
Other external and internal monitoring
The audit team reviewed the list of different reports that relate to policy and program
design options. They found that there are a large number of reports produced on a
regular basis.
On its website, the CRTC publishes a Summary of Local Radio Programming, which
identifies, for those radio stations selected, the amount of local programming provided by
the stations. The staff reviewing the radio content reports stated the reports were meeting
the CRTC responsibility to monitor whether or not stations were meeting Canadian
content requirements.
The CRTC Broadcasting Policy and Monitoring Report and the Telecommunications
Monitoring Report were combined into the Communications Monitoring Report. This
CRTC Operational Audit 25
report provides a more holistic view of the industries and their markets. The report
expands on the performance indicators and trends reported in previous broadcasting and
telecommunications monitoring reports. The audit team concluded, based on interviews
with managers and a review of the report contents, that it meets their needs.
The Web Trends report published internally monitors how the CRTC website is used. It
identifies the number of users visiting the CRTC website, average visits per visitor,
average visitors per day and average visit duration.
A website broken links report is produced on a weekly basis by the Strategic
Communications and Parliamentary Affairs sector. There were approximately 80,000
broken links before the Broken Link Report was produced. A little over one year of
using the report has resulted in a significant reduction in broken links.
The CRTC produces a quarterly report for broadcasting applications in accordance with
Broadcasting Circulars CRTC 2006-1 and 2006-2. Both of these circulars indicate that
the CRTC will post quarterly statistics on its website to allow evaluation of its
performance in meeting the service standards. Auditors observed that the quarterly
reports for April 1, 2008 to March 31, 2009 were published on the CRTC website in
September 2009. Since the quarterly reports on broadcasting service standards are not
regularly published in a timely manner, the audit team concludes that they are not being
used effectively because the information is dated. Although the CRTC has not published
this information on a quarterly basis, the audit team found that the CRTC meets its
obligations to report to the TBS on service standards for external fees annually through
the Departmental Performance Report.
Telecommunications service standards and performance measures for Type 1 and 2 and
local forbearance Part VII applications are currently monitored pursuant to Telecom
Circular CRTC 2006-11, and Telecom Decision CRTC 2006-15, as amended by Order in
Council P.C. 2007-532, 4 April 2007, respectively. The service standards and
performance measures for (a) tariff applications and intercarrier agreements, and (b)
applications regarding the destandardization and/or withdrawal of a service processed
during the fiscal year are also monitored pursuant to Telecom Circular 2006-11.
The Telecommunications Branch has worked with the CRTC IM/IT team in recent
months to develop systems and tools to allow (1) for the efficient tracking of performance
related to the processing of all applications received from the industry, and (2) for the
production of timely reports at the end of each reporting period. As of September 2009,
the CRTC website sets out statistics related to its Telecommunications service standards
and performance measures in a single report, with the statistics being presented in a clear,
concise, and user-friendly format. The report allows the public to monitor the ability of
the CRTC to deal with all applications received in a timely manner. The system tools
and the report allow CRTC management to track and analyse its performance with
respect to the processing of the different types of applications.
The CRTC has complied with Telecom Circular CRTC 2006-11 by posting service
standard results on the CRTC website following the March 31, fiscal year-end. However,
CRTC Operational Audit 26
the CRTC also indicated in Telecom Circular CRTC 2006-11 that it expected to post
service standards on a quarterly basis for information purposes. Posting quarterly reports
on the CRTC website would improve the effectiveness of information provided to
stakeholders and harmonize this reporting function with that of the Broadcasting Sector.
Recommendation:
The Executive Directors of Broadcasting and Telecommunications, in
collaboration with the Executive Director of the Policy Development and
Research, should ensure that the quarterly reports on service standards are
produced on a timely basis. Should reports be delayed, then the CRTC website
should provide a notice concerning the delayed reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential negative publicity regarding the
CRTC business processes.
Management Response:
Response Office of
Primary
Interest
Timeline
The CRTC has an obligation to report on an
annual basis, as set out in the Treasury Board
Policy on Service Standards for External Fees.
CRTC policies referring to quarterly reporting
have since been superseded by more recent
developments. While increased reporting
frequency is desirable, resource limitations
have led to the elimination of quarterly
publication of service standard reports. The
latest annual reports include quarterly
information and can be found on the CRTC
website. Regardless, contracts for the
automation of the reporting process have been
issued. Such automation may permit a return to
a quarterly reporting frequency in the future.
Executive
Directors
Ongoing
Interface with the Stakeholders
Conclusion
Generally, the audit team concluded that the information the CRTC provides to its
regulatory stakeholders is adequate; the CRTC is actively addressing stakeholders’
CRTC Operational Audit 27
comments; and the CRTC has an effective stakeholder consultation process to set and
adjust annual plans.
Website information provided to stakeholders
The CRTC website provides complete and timely information to regulatory stakeholders
with respect to decisions, notices, orders and information bulletins as well as information
on CRTC annual pans and priorities. It also provides the contact coordinates (i.e.
telephone number/e-mail address) to reach Broadcasting and Client Services
representatives with questions, suggestions and complaints.
The audit team observed that the telecommunications single point of contact, put in place
in May 2009, does not appear on the website. The Broadcasting and Client Services
central points of contact are displayed on the website and provide easy access for the
stakeholders to contact the CRTC. The Telecommunications stakeholders who do not
know the Telecommunications single point of contact number can contact the numbers
displayed on the website and their calls are redirected to the appropriate person. Since
the CRTC has provided the coordinates on its website for the other two functions it
would be beneficial to provide the stakeholders with direct access to the
telecommunication function.
CRTC has put in place an internal data base tracking system (i.e. Rapids), to manage
enquiries, complaints or suggestions related to Broadcasting, Telecommunications and
Client Services enquiries, but is only tracking the Broadcasting and Client Services
enquiries. The system is not used to track the telecommunications enquiries.
Recommendation:
The Director of Public Affairs should ensure that the telecommunication
coordinates are published on the CRTC website to allow easy access by the
stakeholders and that a logging system to catalogue calls and responses be
implemented.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential to reduce stakeholder
accessibility and the CRTC ability to
monitor and assess inquiries.
Management Response:
Response Office of
Primary
Interest
Timeline
Coordinates for the Single Point of Contact for
Small Telecom Service Providers have been
posted on the Telecommunications Sector page
Director of
Public Affairs
December
2009
CRTC Operational Audit 28
of the CRTC website.
Work is currently underway to integrate the
operations of the Single Point of Contact for
Small Telecom Service Providers with the
Rapids case management system.
Director of
Public Affairs
April 2010
External service standards
The audit team reviewed the CRTC website, the Departmental Performance Report, the
Public Notice Circulars, the Treasury Board Policy on Service Standards for External
Fees and conducted interviews with managers responsible for service standards.
Treasury Board’s Policy on Service Standards for External Fees requires that
organizations involved in charging external fees must involve the stakeholders in
establishing service standards and must report annually to TBS on the performance. The
audit team noted that the stakeholders were involved in establishing the service standards
and the CRTC reports on the service standards for processing both broadcasting and
telecommunications applications through its Departmental Performance Report. The
CRTC uses Broadcasting and Telecommunication Circulars to inform the stakeholders of
the service standards and report on their achievements on the CRTC website.
Internal service standards
The CRTC has developed service standards for responding to consumer questions and
complaints. The standards are on the CRTC website. The CRTC manually monitors
these service standards to ensure compliance, but does not publish service standards
reports due to the limitation of the Rapids system to perform an age analysis. In order to
produce aging reports manual intervention is required for which resources are not
available. While there is no TB requirement for reporting on these service standards, TB
suggests that service standards performance monitoring is a best practice for government
managers. The absence of reporting on service standards reduces stakeholders’ visibility
of the effectiveness of the CRTC to address questions and complaints.
Action taken to solicit and address stakeholder comments
The audit team concludes that theCRTC is actively soliciting and addressing
stakeholders’ comments. Stakeholders’ comments have been addressed through the
Future Direction initiative and in various Public Notices.
The Operations Committee decided at its meeting of J uly 15, 2009 to defer the industry
consultations to September 2010 where industry and CRTC representatives meet face-to-
face to discuss their priorities. This decision was made because the CRTC did not want
to burden the industry with more consultations. A number of consultations and
communications with industry had occurred over the past few years: consultations had
been undertaken in fiscal year 2007/08; the 2007 Outcomes Report; the May 2008
CRTC Operational Audit 29
External Survey results; and, the TV, Radio, Accessibility Hearings undertaken in the
past two years. Also contributing to this decision was the current economic situation, the
scheduled conventional TV hearing for September 2009 and the fact that the industry was
questioning the necessity for further consultations at this time. Compensating for the
deferral of the annual consultations is the practice of the CRTC staff to have regular
meetings with industry representatives throughout the year, as well as CRTC speeches to
industry asking questions with respect to its priorities. Although not as formal, the needs
and future direction of the industry have been communicated to all those involved in the
planning process.
The audit team reviewed the consultation reports in order to map the stakeholders’
suggestions/recommendations to appropriate action plans to determine how they were
dealt with. The audit team conducted interviews with the responsible managers to
confirm that the stakeholders’ inputs are being addressed. It also met with the heads of
the Future Direction Task Forces to determine if stakeholders’ comments and suggestions
are being reflected in their action plans.
Stakeholders’ comments are captured and communicated to relevant parties through
stakeholder consultations and reports that are internally or externally initiated such as the
Telecommunications Policy Review Panel 2006 report (externally initiated), the2007
Outcomes Report (internally initiated) and the External Communication Study – June
2008 (internally initiated). The consultations are substantial undertakings. For example,
the 2007 Outcomes Report was the culmination of three roundtable consultations with
industry fee payers in Montreal, Toronto and Ottawa to discuss a proposed increase to the
CRTC operating budget for a period of five years. The J une 2008 External
Communication Study was a three-pronged effort including telephone interviews with
1,303 Canadian adults; group discussions in Vancouver, Calgary, Toronto, Montreal,
Quebec City and Halifax; and interviews with 29 senior representatives of CRTC
stakeholder groups.
Stakeholders’ comments and input are also captured on an on-going basis through central
points of contact. In addition, the Media Division scans daily news articles to keep
CRTC staff informed of news events impacting the CRTC. Questions and complaints
received from the public via telephone calls and correspondence (i.e. e-mails and letters)
are recorded in Rapids and tracked to ensure they are addressed.
Although we have not seen any evidence of the individual suggestions/recommendations
being tracked and implemented, we have noted that they are being addressed by the
appropriate Future Direction Task Forces for action. For example, suggestions on
improving the website have been addressed by the Outreach Task Force.
The suggestions and recommendations contained in the above noted reports are not
individually managed because of the work involved in tracking each recommendation.
The audit team noted that the main subjects raised by stakeholders can be characterized
as: policy review and deregulation; more timely decisions; and an improved website so
that stakeholders could find information easier.
CRTC Operational Audit 30
The audit team also noted that actions taken on stakeholders’ recommendations are found
within a number of communication vehicles (i.e. decisions, notices, bulletins, speeches
and initiatives). This fragmentation reduces the overall visibility of CRTC
accomplishments. Considering that the vast majority of regulatory stakeholders rely on
the CRTC website to obtain information, CRTC could regroup achievements and planned
activities under one easily accessible area on the website. Although the website provides
a link to speeches and various reports containing messages by the Chairman and
accomplishments, it does not provide a calendar of future public appearances of the
Chairman, Commissioners and senior managers nor a link to key messages from
management to stakeholders. Improving the transparency of initiatives and achievements
on the CRTC website would help stakeholders to better understand CRTC needs and
challenges.
Recommendation:
The Director of Public Affairs should develop a website page and/or link(s) that
report on achievements, the status of stakeholders’ recommendations provided in
the surveys and reports, and a link to a calendar of future public appearances by
the Chairman, Commissioners and members of the senior management team, as
well as a link to key messages from management contained in various reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that the public is not fully aware
of the CRTC achievements.
Management Response:
Response Office of
Primary
Interest
Timeline
The CRTC website is our number one
communication tool. The diverse nature of
visitors to the CRTC website makes it quite
difficult to have a single webpage that would
contain all of the suggested information and be
relevant to all users. With the exception of a
calendar of future appearances, most of the
identified information can indeed be found in
the “Media” section of the CRTC website. All
speeches and news releases are posted to the
website and contain our key messages. CRTC
achievements are also communicated through
the Report on Plans and Priorities and the
Departmental Performance Report. Important
regulatory information, such as decisions and
Director of
Public Affairs
Ongoing
CRTC Operational Audit 31
information bulletins, has been organized by
industry and all documents can be accessed
easily and efficiently from a number of
redundant links. Finally, a dedicated
“Consumer” section exists for key information
of particular interest to the general public.
A calendar of future appearances has not been
developed as the sensitive nature of the
Commission’s quasi-judicial role in the
industry can often necessitate last minute
changes in attendance due to a variety of
potential conflicts. The Director of Public
Affairs will however examine this proposal in
the new fiscal year with a view to providing as
much information to the public as possible.
Director of
Public Affairs
FY 2010/11
Setting & adjusting annual plans through stakeholder consultation
The audit team concluded that the CRTC has an effective stakeholder consultation
process to set and adjust annual plans.
The audit team reviewed the 3-Year Work Plan and the annual Report on Plans and
Priorities (RPP) and interviewed key personnel involved in the development of plans and
public consultations.
The CRTC undertakes a variety of consultations with stakeholders to set and adjust
annual plans. The results of the consultations are reflected in a 3-Year Work Plan and the
annual Report on Plans and Priorities (RPP). Specifically, the CRTC held industry
consultations in J anuary 2008, where industry and CRTC representatives met for a three-
day session to discuss their priorities. Other sources of information utilized by the CRTC
to determine priorities included the Outcomes Report – 2007, External Survey results –
May 2008, and TV, Radio and Accessibility Hearings undertaken in the past two years.
In addition, CRTC staff has regular meetings with Industry representatives throughout the
year to discuss priorities and annual plans, and to seek input and the Chairman delivers
speeches to the industry on licence renewals, Canadian programming, and national digital
strategy.
Interface between the Sectors
Conclusion
The audit team concludes that there is ongoing and transparent communication between
the Operations Committee (i.e. oversight body), management, Commissioners and the
CRTC Operational Audit 32
Chairman resulting in an effective interface between the sectors in support of enhanced
regulatory processes and results. This was demonstrated by the existence of an effective
internal governance structure that incorporates a number of committees that require the
involvement of the Commissioners and staff to address regulatory and management
issues. During audit interviews managers and staff indicated that there was appropriate
interface between sectors, within sectors, with Commissioners and with the functional
groups (i.e. legal, finance, human resources and communications).
Governance
The governance structure in place ensures that management’s direction, plans and actions
are appropriate and responsible. Several committees have been established to deal with
management and regulatory responsibilities. A key committee which looks at operational
and management issues is the Operations Committee. This committee meets on a weekly
basis and is comprised of the Chairman, Vice Chairman Telecommunications, Vice
Chairman Broadcasting, Secretary General, Executive Director Broadcasting, Executive
Director Telecommunications, Executive Director Policy Development and Research,
Senior General Counsel and the Director General Strategic Communications and
Parliamentary Affairs. In addition to the Operations Committee, weekly bilateral
meetings are held between the Chairman and the senior manager of each sector to discuss
operational and policy issues specific to that sector (i.e. Secretary General, Legal,
Strategic Communications, Broadcasting, Telecommunications and PDR). The
Operations Committee and the sector bilateral meetings produce summary and follow-up
documents that record decisions made, instructions provided and follow up required.
These records of decisions demonstrate management’s interventions into reallocation
decisions to support the regulatory processes and results with the resources that the
Commission has been provided.
Several other committees have been established to deal with management responsibilities
that support the Operations Committee. These include the IM/IT Steering Committee,
the HR Committee, the Steering Committees on Future Direction (i.e. streamlined rules
and regulations, effective outreach and succession/continuity planning), the Diversity and
Employment Equity Committee and the Commission’s Organizational Health and Safety
Committee. Each committee draws its membership from across the organization to
ensure all sectors are represented.
The organization’s regulatory responsibilities are discharged by a number of committees
involving the Commissioners and staff to address broadcasting and telecommunications
related issues. These committees include the Commission Full-time Members Meetings
(C/FTM), the Broadcasting Committee Meeting (BCM), the Broadcasting Committee
Meeting – Walk Around (BCM-WA), the Broadcasting Committee Review (BCR), the
Telecommunication Committee Meeting (TCM), the Telecommunication Committee
Meeting – Walk Around (TCM-WA), the Telecommunication Committee Review (TCR),
and the Senior staff members +Commissioners i.e. Operations Committee with
Commissioners (SSM+).
CRTC Operational Audit 33
The C/FTM and SSM+are chaired by the Chairman, and all Commissioners and senior
staff attend and discuss key issues. The BCM, BCM-WA, TCM and TCM-WA meet
weekly, or more frequently as required, and are chaired by the Vice-Chairperson of
Broadcasting and Telecommunications, respectively. The BCR and TCR are broadcasting
and telecommunication staff meetings and occur weekly to review incoming applications.
Both management and regulatory committees have terms of reference articulating
membership, meeting times and purpose. Whenever appropriate, committee membership
is drawn from all sectors resulting in equitable representation from across the
organization.
In order to meet the operational demands of the regulatory processes a matrix
management approach is utilized enabling groups to draw resources from other sectors to
meet both pressing and unexpected demands. For instance, the Policy, Development and
Research (PDR) Sector will utilize resources from the other sectors to participate in
research and policy development initiatives.
Perhaps the most demanding initiative requiring effective interface between the sectors is
the Future Direction initiative. The four working groups established encompass all
sectors of the CRTC by drawing resources from each sector to develop work plans and
identify key priorities. In addition, a communication plan was developed to ensure
everyone in the organization has access to information about the status of each working
group’s action plans for implementation, as well as opportunities to voice their own
concerns, questions and views.
Staff in the Legal Directorate, Finance and Administrative Services, Human Resources
and Strategic Communications is involved in regulatory and management responsibilities
through their participation in the committees established to deal with these
responsibilities. As well as their involvement in the Future Direction initiative, as noted
above, members of these sectors are also effectively interfacing with each other in their
day-to-day regulatory and management responsibilities. For instance, Finance is working
with the sectors to implement the enhanced salary management system, HR is working
with individual sectors to update HR Strategies, Strategic Communications is working
with sectors to implement the enhanced Performance Measurement Framework and the
Legal Directorate is working with other sectors in the execution of its legal
responsibilities as well as its role in the streamlining of rules and regulations.
Internal communications
The audit concluded that open and effective channels exist for internal communications
and feedback. The CRTC management has undertaken a number of initiatives to identify
and address employee concerns. This is evidenced by the internal communications
throughout the Future Direction exercise, the revised plan for the Future Direction
implementation phase, the Internal Communication Survey, focus group evaluation and
complete revamping of the Zone, as well as the recent development of an action plan to
address the results of the 2008 Public Service Employee Survey (PSES).
CRTC Operational Audit 34
Senior management regularly communicates and encourages ongoing dialogue amongst
employees. The CRTC utilizes the Zone to support the dissemination of relevant
information to employees. There is a regular newsletter (The Frequency) for all
employees to view and Weekly News that contains upcoming events, human resource
and finance news, community news, weekly messages on various topics, Chairman’s
messages, etc. In addition, strategic documentation is also available on the Zone (e.g. 3-
Year Work Plan, Future Direction initiative, HR Strategic Plan, etc.). As a result,
employees have access to the mandate, direction and priorities of the CRTC. HR and
Finance work closely with Internal Communications on a variety of HR/Finance
employee and management issues. With respect to operational and management
responsibilities, each sector participates in bi-lateral meetings with the Chairman and
other sector representatives are invited to attend as observers. This was initiated under
the present Chairman and has been well received as noted in the various interviews the
audit team has had with employees.
An Internal Communications Plan for the CRTC Future Direction initiative was
developed for fall 2008/winter 2009. The purpose of the plan was to help senior
managers continue to communicate effectively with CRTC staff – in a transparent,
diligent and coherent manner – regarding the next steps in the Future Direction strategic
exercise. As recommended in the report under the Future Direction initiative section,
progress reporting should be done in a more complete and timely manner.
Management Control Framework
Conclusion
The audit team concludes that the main elements of the Management Control Framework
(MCF) are in place and effective. This was demonstrated by strong strategic planning that
is aligned with the CRTC mandate, an enhanced performance measurement framework
that will improve DPR, the key planning elements (i.e. performance appraisals, review of
year end results, setting corporate objectives, consultations, RPP and budget approval)
having been included in the CRTC planning cycle, and strong Financial and Human
Resource management.
The audit team noted that the responsibility centre managers are not assuming their
responsibility for financial monitoring, project management should be better entrenched,
progress reporting for the Future Direction initiative should be enhanced to provide more
complete and timely reporting, and the business planning and reporting process should
combine all the core planning and reporting elements into one formal integrated process.
Background - incremental funding and SDA requirements
The CRTC is 100 per cent cost recovered through Part I Broadcasting Licence Fees and
Annual Telecommunications Fees. These fees are the only available source of funding to
address its on-going and incremental budgetary requirements. The CRTC budget had
CRTC Operational Audit 35
CRTC Operational Audit 36
been stable for a number of years. In 2007 it undertook Fee Payer Consultations and
proposed an increase to its operating budget for a period of five years. The incremental
funding was to be directed at the streamlining of processes and policies, reduced
regulatory burden and increased timeliness of service to the industry and the public. As a
result of the consultations, the CRTC was provided incremental funding for two years
(i.e. fiscal years 2007/08 and 2008/09). In fiscal year 2009/10, the incremental funding
was discontinued and CRTC was at its pre-fee payer consultation funding level.
The CRTC falls into the category of a small department
2
as established by Treasury
Board. The Auditor General recently examined governance of small federal departments
and agencies in Chapter 2 of a report tabled in early February 2009. The issue of a heavy
administrative and reporting burden was raised and two recommendations were made
with respect to easing the reporting burden and facilitating administrative shared services.
TheCRTC, like many small agencies, does not have the capacity to respond to the
reporting requirements of the government’s central agencies and is awaiting the TBS
development and implementation of an action plan to reduce the reporting burden and
facilitate a framework for shared corporate services. Until this is completed, theCRTC
must continue to leverage its capacity between the demands of its core mandate while
satisfying the reporting requirements under the reporting regime of central agencies.
The termination of the CRTC incremental funding combined with its limited capacity to
respond to the reporting requirements of the government’s central agencies further
exacerbates a situation in which the CRTC finds itself, straining to deliver its mandate in
a changing and complex environment.
During the audit engagement, staff conveyed to the audit team a feeling of being
overwhelmed by both the demands of the profound changes in the industry and the
demands of the central agencies reporting requirements. In order to maintain excellence
in the delivery of their mandate, the reporting requirements demands were neither a high
priority nor of the standard that they would like to achieve.
Strategic direction and performance measurement
The organization has a clearly defined and communicated strategic direction that is
aligned with its mandate. The CRTC 3-Year Work Plan (2008-2011), the 2009/10
Report on Plans and Priorities (RPP) and Program Activity Architecture (PAA)
adequately address the organization’s strategic outcome, priorities and related action
plans. These documents identify three priorities and articulate the planned actions over a
three year period (i.e. 2008/09, 2009/10 & 2010/11).
The CRTC has been working with Treasury Board officials to develop a more
comprehensive and cohesive program description to better reflect its overall mandate. As
a result of last year’s Management, Resources and Results Structure (MRRS) review, the
CRTC has revised its current Performance Measurement Framework (PMF). The
2010/11 RPP will have one strategic outcome, three output statements, six expected
2
Fewer than 500 employees and annual expenditures of less than $300M.
results and 16 performance indicators for the three program activities (i.e. broadcasting,
telecommunications and internal services). Under the new PMF, the CRTC will review
its performance targets mid-year and third quarter as part of the financial and non
financial review. In addition, performance measures will be reviewed on an annual basis
as part of the RPP planning process.
The audit team reviewed the revised 2010/11 PMF and found that the two weaknesses
identified by Treasury Board officials had been addressed. The CRTC revised its
performance indicators to better measure the progress towards its Strategic Outcome
(SO) and it has developed and implemented performance indicators to measure the output
levels. Performance indicators have increased from 5 to 16 for fiscal year 2010/11 and
should improve the organization’s ability to report performance in future Departmental
Performance Reports (DPR).
Planning cycle
The audit team reviewed the key elements of the CRTC planning cycle (i.e. performance
appraisals, review of year end results, setting corporate objectives, consultations, RPP
and budget approval). It found that the employee performance appraisals and year end
results were evaluated and the results were incorporated into this year’s employee
objectives. Although employee objectives had not yet been formally approved by the
Chairman at the time of the audit, the audit team was satisfied that the draft objectives
were reflective of the CRTC strategic direction for the year.
As noted above, the CRTC has been working with Treasury Board officials to develop a
more comprehensive and cohesive program description to better reflect its overall
mandate. A revised PMF has been developed and is being implemented in the RPP
which should improve the organizations ability to report performance in future DPR.
The budget approval process for FY 2009/10 was very rigorous. FY 2008/09 was the last
year for CRTC incremental funding; therefore, an A-base review process was required to
establish both salary and operations & maintenance (O&M) funding for the coming fiscal
year. A number of salary scenarios including the financial impact were presented to the
Operations Committee. The basic objective was to allow the organization to carry out its
mandate and minimize the risk of job loss. In the end, the scenario that allowed the
CRTC to meet both these objectives was selected and a notional salary budget allocation
was set by the Operations Committee at its J anuary 26, 2009 meeting. For the O&M
funding, costing information based on actual expenses for the past three years by
responsibility center and sector was prepared and distributed to each sector. Each sector
was asked to identify the amount they considered necessary as a base operational and
maintenance allocation. The total of all sectors could not exceed the overall set base. A
number of budget meetings held between finance and the sectors ensued, and finally at
the Operations Committee meeting of May 19, 2009 the operating budget, including both
salary and O&M funding allocations was approved.
CRTC Operational Audit 37
The audit team concludes that activities and resources to achieve the organization’s
objectives have been integrated into the budget, a formal process is in place to review and
set resource allocations, and a timely budget is developed at the appropriate level of
detail.
Business planning and reporting process
There is no integrated business planning and reporting process that combines all the core
planning and reporting elements into one formal process. The audit team found that
many of the planning and reporting elements are available individually; however, they
are not collectively brought together through the application of an integrated business
planning and reporting process. The audit team found that the current business planning
and reporting process is effective; however, the efficiency of the process can be
enhanced.
The current planning process consists of the 3-Year Work Plan, EX Performance
Agreements, Reports on Plans and Priorities (RPP) and budget and resource allocation.
The reporting process consists of periodic reviews of financial information, Departmental
Performance Report (DPR), EX performance reviews, employee performance feedback
and employee learning plans.
To ensure that the output of one process becomes the input to the next and little or no
duplication of effort occurs within the planning community and individual sectors, the
business plan should include both the IT and communication strategies and associated
costs, the performance measurement framework targets and planned mid-year and third
quarter reviews, potential sources of funding and potential risks that could negatively
impact the CRTC’s strategic direction. By including all of the above in the business
planning and reporting process, each item would be assessed throughout the year as part
of the financial and non financial reviews by the Operations Committee. This would
ensure that the status of the planning elements are periodically reviewed and discussed at
the Operations Committee meetings.
To facilitate the periodic discussions a pre-set forward agenda could be developed to
ensure all major planning and reporting elements are reviewed at specified times
throughout the year. The implementation of a pre-set forward agenda that both identifies
and sets a time for pertinent planning and reporting elements (i.e. IT/IM Strategy,
Communication Strategy, HR Strategy, performance targets, funding and risk
management, EX Performance Agreements and performance reviews) to be reviewed by
the Operations Committee would ensure that all planning and reporting elements are
considered at the appropriate time for discussion and decision, thereby ensuring that the
output of one planning element becomes the input to the next planning element process
with little or no duplication of effort.
The audit team recognizes that the strategic plan must be very dynamic to meet the
demands of external influences which can change over night. As an example, an
unplanned public hearing process can be initiated for an issue that was not foreseen only
CRTC Operational Audit 38
months earlier. The CRTC identifies and intuitively assesses risks and mitigating
strategies for management decisions and regulatory planning with respect to its three
program activities – broadcasting, telecommunications and internal services; however,
they are not collectively documented and discussed in the business planning and
reporting process.
Presently the business planning and reporting process does not identify and document
possible risks and associated mitigating strategies such as hiring temporary staff,
reducing service levels, etc. to offset any impact on staff. TheCRTC must react to these
occurrences and the consequences of their decision may not be readily known and
understood fully without previously assessing the risk and documenting the mitigating
strategies. Mitigating strategies would consider the impact on all other activities that
could be impacted by unplanned hearings. Without the road map of a comprehensive
business planning and reporting process, senior management will not always be fully
equipped to assess the impact on already committed resources and priorities.
An integrated business planning and reporting process that combines all the planning and
reporting elements will enable senior management to fully assess financial and non
financial information and the impact on all CRTC activities, thereby maintaining a
strategic focus throughout the year.
Recommendation:
The Director General of Strategic Communications and Parliamentary Affairs,
in collaboration with the Secretary General and the Executive Directors of
Broadcasting, Telecommunications and PDR, should develop and implement an
integrated planning and reporting process and establish a pre-set forward
agenda to ensure that all major planning and reporting elements are reviewed at
specified times throughout the year.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential that senior management will not
maintain a strategic focus throughout the
year.
Management Response:
Response Office of Primary
Interest
Timeline
A CRTC Annual Corporate Planning and
Reporting cycle has been developed in
consultation with all sectors of the CRTC. It
will be presented for the approval of the
Operations Committee.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010
CRTC Operational Audit 39
Development and approval of the cycle is
the first phase in establishing an integrated
planning and reporting process throughout
the organization. Additional processes will
be identified and incorporated into the
process over the next few months. (For
example, further input from members of the
Corporate Management Committee will be
solicited in J anuary 2010.)
Director General,
Strategic
Communications
and Parliamentary
Affairs
FY 2010/11
Financial management/monitoring
Financial management policies and authorities are established and communicated.
Policies are available on the intranet and/or links are made to the TB site for financial
policies. Financial policies and authorities are known and understood.
Responsibility for financial monitoring is included in each manager’s Performance
Management Agreement. Each Executive Director has a financial management
commitment with appropriate performance measures (i.e. budget allocations are aligned
with CRTC priorities; commitments are updated on a monthly basis; and expenditures are
monitored monthly and do not exceed available budget allocations). Each sector head
and their management team are now fully responsible and accountable for all aspects of
budget management for their sector. Finance pre-approval for job requisitions is no
longer required, as it was in fiscal 2008/09. It is now the responsibility of each sector
head to ensure that all staffing actions undertaken are within their approved salary budget
and that salary forecasts are updated on a monthly basis. To assist managers to achieve
the performance measure, the Finance Division has established procedures and provides
reports to each Executive Director, and financial staff members assist each sector on a
monthly basis.
The first quarter of fiscal year 2009/10 was not completed on time by the managers since
this was a transition year where the salary budget shifted from the Finance Directorate to
Responsibility Centre Managers. Rather than present an unrealistic budget summary to
the Operations Committee, the Finance Directorate decided to defer the J uly 31, 2009
first quarter reporting to allow additional time for Responsibility Center Managers to
input the required commitments. As a result, the budget summary report presentation to
the Operations Committee occurred in August 2009, resulting in a time span of almost
five months before senior management was provided with an overview of the CRTC
2009/10 spending.
This time span is in direct contrast to fiscal year 2008/09, where the financial reporting
package was presented to the Operations Committee less than 15 calendar days after
period end and was presented to management ten or more times per year. The timeliness
issue is likely attributable to the shift of responsibility from the Finance Division to the
CRTC Operational Audit 40
Executive Directors who may be experiencing problems as they implement the new
practice with respect to salary and O&M budget allocations and management.
Recommendation:
The Director General Finance and Administrative Services should continue to
reiterate to the Responsibility Centre Managers their responsibility for financial
budgeting and monitoring to ensure that they understand and provide complete,
accurate and timely financial information for the monthly budget summary
report.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential that financial information
reported to senior management is not
complete and timely.
Management Response:
Response Office of Primary
Interest
Timeline
Procedures are in place whereby Finance
staff advises sector heads of the requirement
to review and update all their commitments
(Salary and O&M) on a monthly basis
before month end. In addition, as part of
these monthly notifications Finance staff
have advised sector heads that additional
financial training to sector management
teams would be provided upon request.
Director General,
Finance and
Administrative
Services
Ongoing
The requirement to do a comprehensive and
monthly review of all financial
commitments was recently re-emphasized
by the Secretary General (Chief Financial
Officer) to the Executive Committee as part
of the FY 2009/10 mid-year budget review.
Secretary General November
2009
Executive performance agreements have
been updated to reflect the expectation that
all members of the Executive Committee
and their management teams will review
and update their commitments monthly.
Director General,
Human Resources
Completed
Human resource management
The audit team concludes that human resource planning is aligned with strategic and
business planning, enabling theCRTC to identify current and future resource
CRTC Operational Audit 41
requirements and analyzes them against existing human resource competencies and
capacities.
The HR Strategic Plan reflects the conclusions and identifies strategies to implement
priorities identified by the CRTC Future Direction Task Force “Innovative Approaches to
People and Succession Planning” initiated in 2007/08. The HR Plan articulates the
following seven key initiatives: build capacity for key leadership positions, build
workforce capacity, maintain and develop current workforce, meet changing business
needs, build and maintain bilingual capacity for key leadership positions, build a diverse
and representative workforce, and renewal. Each initiative is broken down into activity,
responsibility, indicators, objective, status and comments and is monitored by HR on a
regular basis. Currently each sector is updating its section of the plan and the overall
results will be presented to the Operations Committee this fall.
Recruitment plans have been put in place that addresses the need to build capacity for the
key EX leadership positions. In May 2009 the EX shortage was addressed with the EX1
and EX2 competition and subsequent staffing actions.
The audit team concludes that the organization has an effective performance management
program for all of its employees. Responsibilities and performance expectations to which
staff are accountable are formally defined and clearly communicated. J ob descriptions
and performance agreements exist for this purpose and are up-to-date. Audit testing
indicated that performance management agreements (i.e. EX Group) and performance
feedback agreements (i.e. Non EX Group) are aligned with the 3-Year Plan and the RPP.
Performance management agreements contain clear and assessable commitments,
including HR and financial management while performance feedback agreements have
attainable work objectives. Performance training plans are established and implemented
with clear accountability and all staff has access to training and support in the
performance evaluation and feedback process. An effective review mechanism is in
place to ensure equity and consistency in determining ratings.
Future Direction initiative
The CRTC executive team, at its November 2007 retreat, discussed its future direction
and how it could rise to the challenges over the next five years – covering its purpose,
core business and key focus areas. The challenges were the result of the profound change
to the communication industry as new digital technologies transform how Canadians
communicate, how theCRTC delivers its mandate and how the public is informed and
entertained.
Four internal task forces were created to consult, conduct further analysis and develop
action plans for the four key areas, each under the leadership of a senior CRTC executive.
These areas are:
• Structures that address convergence;
• Focused and streamlined rules and regulations;
CRTC Operational Audit 42
• Achieving effective outreach; and,
• Innovative approaches to people and succession issues.
Each task force prepared an action plan that was presented to the Operations Committee
in J une 2008 and presented to all CRTC employees at an all-staff meeting on J uly 12,
2008, after which each task force commenced working on its assigned area of focus.
Future Direction was a major internal initiative that further increased demands on staff
time. The additional workload was completed without staff augmentation and in addition
to the demands of their day jobs.
During the audit team’s initial interviews with managers, they expressed a concern that
the Future Direction initiative had lost momentum. During the examination phase of the
audit, the team assessed whether this was caused by inadequate communication, team
leads not providing the status on a regular basis or overall project management
responsibilities not clearly articulated and assigned.
a) Communication
A review of the Zone indicated that there had been a number of communications on the
Future Direction (FD) initiative. They were as follows:
• April 9, 2009 - FD Status Report on Goals & Milestones;
• Winter 2009 – Quarterly Reports from the Champions;
• Fall 2008 – Quarterly Reports from the Champions;
• October 2008 – Succession planning initiative;
• September 2008 – Continuity Plan; and,
• J uly 2008 – Conclusions from retreat presented to employees since the initiative
began in May 2008.
Since the Town Hall meeting in J uly 2008, the Zone indicated that six communications
on the initiative had been published; however, the audit team noted that the “Winter 2009
- Quarterly Report from the Champions” had not been published. The audit team was
told that the March 12, 2009 update to the Operations Committee included an update of
only one key area, “Innovated approaches to people and succession planning”. The audit
team was informed that it was not published on the Zone because of an oversight. The
audit team reviewed the update briefing material provided to the Operations Committee
and concluded that this was correct because it was a good news story. The Operations
Committee was not provided updates for the other two key areas still active (i.e. Outreach
and Streamlining).
The audit team concluded that the communication of the Future Direction initiative could
be enhanced to ensure it is complete and timely.
CRTC Operational Audit 43
Recommendation:
The Champions, in collaboration with their project leads and the Director
General of Strategic Communication and Parliamentary Affairs, should ensure
that the Quarterly Reports on Future Direction are consistently published in a
complete and timely manner.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that incomplete reporting may
cause a decline in employee support for
the initiative.
Management Response:
Response Office of Primary
Interest
Timeframe
The Director General of Strategic
Communications and Parliamentary Affairs
will reiterate the importance of quarterly
reporting to the project leads and ensure that
all information is provided in a complete and
timely manner.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010 and
ongoing
Publication dates for quarterly reports will
be pre-set and the Strategic Communications
sector will ensure that all required
information is submitted in conformity with
the relevant timelines.
Director General,
Strategic
Communications
and Parliamentary
Affairs
April 2010
and ongoing
b) Project management
The audit team reviewed the action plans developed for each of the four key areas and
interviewed responsible managers to assess the progress since the action plans were
presented at the all-staff meeting on J uly 12, 2008.
i) Key Area - Structures that address convergence
The key area had been implemented shortly after the all-staff meeting. It was a critical
element that needed to be completed first to provide a solid organization structure to
move ahead with the other three key areas. Activities common to both Broadcasting and
Telecommunications became the responsibility of the new sector Policy Development
and Research (PDR) and organization charts were restructured.
The audit team concluded that this key area has been successfully implemented as
evidenced by interviews with responsible managers and staff, and review of pertinent
documentation.
CRTC Operational Audit 44
ii) Key Area - Innovative approaches to people and succession planning
The key area is comprised of the following five activities: succession/continuity
planning; job standardization; job rotation; leadership training; and the “CRTC
University” (CRTC-U).
The majority of tasks under the succession/continuity planning activity are completed.
Key leadership positions have been identified and staffing for EX-01 and EX-02
positions has been carried out. The talent management plans for EX, EX equivalent and
EX minus one groups have been developed and were reviewed as part of the annual
performance review. The talent management exercise will be extended to all other
employees, and managers will be responsible for establishing development plans with
their employees as well as ensuring continuity within their own operations. At the time
of the audit it was expected to be completed in fall 2009.
A consultant had been selected to create a customized leadership development program
for CRTC managers that will focus on people skills and people management. It is
expected to be ready for delivery in fall 2009.
The CRTC-U training program includes an orientation program for new employees as
well as a development program for existing employees. A sub-committee of CRTC
employees was struck and it is expected to roll-out CRTC-U in fall 2009.
Although not all activities have been implemented, the audit team concludes that this key
area is making effective progress and is adequately managed as evidenced by interviews
with responsible managers and staff, and review of pertinent documentation.
iii) Key Area - Achieving Effective Outreach
The key area is comprised of three activities: engage and educate smaller groups in the
industry; create a better understanding of the CRTC mandate and activities for members
of the government and the public; and, create an awareness of the CRTC with the 18 to
34 year age groups. Portions of the activities are either completed or works-in-process.
Budget concerns are impacting some of the activities.
Although not all activities have been implemented, the audit team concludes that the key
area is making effective progress and is adequately managed as evidenced by interviews
with responsible managers and staff, and review of pertinent documentation.
iv) Key Area - Streamlined Rules and Regulations
Prior to the Future Direction initiative a government policy direction was issued in
December 2006 to foster increased reliance on market forces in the telecommunications
CRTC Operational Audit 45
industry. The directive led to the initiation of a number of activities within the CRTC
Telecommunications sector. Separately under the Future Direction initiative, additional
activities were initiated relating to a number of process improvements and changes in the
regulatory framework, including the removal of some existing rules.
These two initiatives triggered a number of activities within the CRTC to address the
common overall objective of a more simplified regulatory process. One of the initial
actions undertaken by the key area project manager was to identify and document all the
activities carried out by the Commission which necessarily captured those generated by
the policy direction. The purpose of the activity catalogue was to identify further process
improvement opportunities. The activities were documented in the activity catalogue and
divided by sector – Telecommunications, Broadcasting, PDR and Corporate Services and
Operations. While the activities initiated within the telecommunications sector that
addressed the implementation of the government policy direction formed part of the
activities catalogue, they continued to be implemented by the telecommunications
directorate, independently of the Future Directions initiative.
The audit team reviewed the activity catalogue and completed a comparison of the
activities recorded in the catalogue to the April 9, 2009 Future Direction Status Report on
Goals & Milestones (i.e. Quarterly Report) published on the Zone. As the April Status
Report dealt only with the activities that were retained as priority items to be addressed at
the Future Direction Workshop held in May 2008 and the Public Hearing Process Charts
prepared by a consulting firm, not all activities initially recorded in the activity catalogue
were included in the Quarterly Report and not all activities underway outside of the
Streamlining initiative were included in the catalogue.
The results of the comparison were discussed with the responsible managers and it was
agreed that activities underway for which no status was provided included 19
telecommunications activities, 8 PDR activities and 15 broadcasting activities. The audit
team interviewed those responsible for the activities not reported and not included in the
catalogue. They stated that these activities were at various stages of completion;
however, they had not been directed to provide periodic updates.
The project manager for the Streamlining initiative reported that when he created the
activities catalogue it was never envisioned that the catalogue would become a living
document which would be updated overtime. Rather it was a tool developed for the
initial stages of the Streamlining project to identify the processes most in need of
improvement.
As a result, only some of the activities being carried out in various parts of the
Commission were found in both the Quarterly Report and the Activity catalogue.
However, the audit team was satisfied that staff responsible for the individual activities
was managing them adequately.
The audit team concludes that the following factors have contributed to employee
concerns that the initiative had lost momentum:
CRTC Operational Audit 46
• The CRTC did not develop a formal monitoring and reporting framework for
activities under the streamlining initiative;
• The activity catalogue was not developed for the purpose of providing centralized
monitoring of process improvements;
• There may be lost opportunities for possible cross-sector integration of common
activities; and,
• The performance objectives of the project manager responsible for the key area
only referred to the combined rules of procedures, which is only one activity of
the key area.
These factors resulted in the lack of complete and timely weekly reporting as set out in
the terms of reference as well as quarterly reporting as set out in the Future Direction
initiative communication plan.
Recommendation:
The Future Direction Champion and Project Manager for Streamlined Rules
and Regulations, in collaboration with the Executive Directors should:
• Identify the activities that constitute the Future Direction
streamlining and policy direction initiatives underway;
• Identify possible activities that can be integrated;
• Update the activity catalogue on a regular basis; and,
• Clearly establish responsibility for the management of the key
area as well as other initiative and ensure that complete and
timely bi-weekly progress reporting is implemented.
Risk Type Audit Risk Rating Impact
Operations
Risk
Minor Potential that not all activities are
effectively managed.
Management Response:
Response Office of
Primary
Interest
Timeframe
Priorities for the Streamlined Rules and
Regulations initiative were initially
established through a Process Mapping
exercise that identified process “hot spots”
most in need of attention. Remaining
activities will continue to be reviewed and re-
prioritized on an ongoing basis according to
operational requirements and resources.
Future Direction
Champion,
Streamlined
Rules and
Regulations
Ongoing
CRTC Operational Audit 47
Project management will collaborate with
Strategic Communications to ensure that
quarterly status updates are issued on a
regular and timely basis.
Project
Manager,
Streamlined
Rules and
Regulations
J anuary 2010
and ongoing
Project scope and management responsibility
will be further clarified through the upcoming
development of Future Direction Exercise -
Phase II.
Future Direction
Champion,
Streamlined
Rules and
Regulations
April 2010
CRTC Operational Audit 48
RISK ASSESSMENT FRAMEWORK ANNEX 1
Risk Types
Audit risk types have been classified according the COSO Internal Control-Integrated
Framework as follows:
Strategy: High-level goals, aligned with and supporting the CRTC mandate
Operations: Effective and efficient use of resources
Monitoring: Accurate assessments or evaluation of activities
Reporting: Reliability of operational and financial reporting
Compliance: Compliance with applicable laws, regulations, policies and procedures
Risk Ratings
Audit findings are rated as follows:
Major: A key control does not exist, is poorly designed or is not operating as intended
and the risk is potentially significant. The objective to which the control relates is
unlikely to be achieved. Corrective action is needed to ensure controls are cost effective
and/or objectives are achieved.
Moderate: A key control does not exist, is poorly designed or is not operating as
intended and the related risk is more than inconsequential. However, a compensating
control exists. Corrective action is needed to avoid sole reliance on compensating
controls and/or ensure controls are cost effective.
Minor: A weakness in the design and/or operation of a non-key process control. Ability
to achieve process objectives is unlikely to be impacted. Corrective action is suggested
to ensure controls are cost effective.
CRTC Operational Audit 49
LIST OF POSITIONS INTERVIEWED ANNEX 2
Position Title
Vice-Chairman, Broadcasting
Vice-Chairman, Telecommunications
Executive Director, Broadcasting
Executive Director, Telecommunications
Executive Director, Policy Development & Research
Senior General Counsel
Secretary General
Senior Advisor to the Secretary General
Director General, Television Policy & Applications
Director General, Policy, Decisions & Operations
Director General, Competition, Costing & Tariffs
Director General , Strategic Communications & Parliamentary Affairs
Director General, Human Resources
Director General, Finance & Administration Services
Director, Convergence Policy
Director, Industry Analysis
Legal Counsel (s)
Director, Planning & Process
Director, IM/IT
Director, Public Affairs
Senior Communications Officer Stakeholder Relations
Director, Dispute Resolution
Director, Communication Policy (Telecom)
Director, Ownership and Acquisitions (PDR)
Director, Decisions and Operations (Telecom)
Senior Director, Alternate Dispute Resolutions
Senior Director, Distribution Policy & Applications
Director, Broadcasting, Streamlining and Decisions
Correspondence & Systems Coordinator
A/Manager, Data Collection and System
Manager, Web Publishing Policy & Standards
Assistant Director IT, Business Solutions Center
Client Relationship Manager
Security Architect
IT Security Coordinator
Programmer/Analyst
Manager, Information Resource Centre
Information Management Policy & Planning Analyst
Manager, Ownership Data & Management
Radio Monitoring Coordinator
CRTC Operational Audit 50
Specialist and Systems Analyst
Director, Dispute Resolution
Director, Communication Policy (Telecom)
Director, Ownership and Acquisitions (PDR)
Director, Decisions and Operations (Telecom)
Director of Media & Parliamentary Relations
Telecom Analyst /Telecom Contact
Manager, Broadcasting Process and External Liaison
Manager, Parliamentary Relations, Corporate Planning and Reporting
Senior Communications Officer, Parliamentary Relations
Chief, Human Resources Operations
Chief, Human Resources Programs & Planning
Human Resource Program Officer
Assistant Director, Financial Policy, Planning & Systems
Senior Financial Planning & Reporting Officer
Special Advisor, Operations
Chief Coordinator, Chairman’s Office
Senior Analyst, Competitive Disputes
CRTC Operational Audit 51
AUDIT OBJECTIVES AND AUDIT CRITERIA ANNEX 3
Audit Objective 1
The effectiveness and efficiency of regulatory processes and practices (e.g. public
hearing processes, application processes, scheduling, etc.), and to ensure that
services are delivered in a fair, open, transparent and timely basis and in a manner
that minimizes the regulatory burden on all stakeholders. This will include an
assessment of the on-going and planned CRTC initiatives to enhance the
regulatory processes and service delivery.
Audit Criteria
1. The organization has resources to support research and policy analysis.
2. The organization has access to adequate subject-matter expertise for the conduct
of policy research.
3. Responsibility for research, consultation and analysis of policy options and
related impacts on program is clearly and formally delineated.
4. A process exists to solicit stakeholder and employee comments related to
improvements in service delivery.
5. Significant change initiatives and management actions are communicated to the
appropriate stakeholders on a timely basis.
6. When the CRTC wishes to develop or change a policy, the CRTC adopts a
public notice describing the proposed changes. A deadline is specified for
comments and reply comments.
7. CRTC proceedings attract participation from opposing interests, so it normally
informs critiques of industry positions.
8. Public interest groups participate in appropriate proceedings.
9. All comments and reply comments are made a part of the public record of the
proceeding.
10. Only in very limited circumstances does the CRTC permit parties to attend its
proceedings to submit confidential material.
11. The CRTC makes decisions public and explains the rationale for their
decisions.
12. Members of the public that wish the CRTC to change its rules or develop new
ones may file appropriate Forms requesting that action. The CRTC puts such
petitions out for comment and reply comment before deciding whether to issue
ruling.
13. The CRTC receives, processes, and assists in the resolution of consumer
complaints. This process is well documented.
Audit Objective 2
The effectiveness of existing information systems that are used to collect and
provide information in support of the CRTC regulatory processes (e.g. the
reliability of information systems for decision-making and accountability
CRTC Operational Audit 52
purposes).
Audit Criteria
1. Records, data and information are appropriately secured in compliance with
privacy legislation.
2. The organization leverages information technology to enhance user service and
access.
3. Transactions are coded and recorded accurately and in a timely manner to
support accurate and timely information processing.
4. Controls are in place to ensure accuracy of transaction coding and processing
(e.g., batch totals, reconciliations, supervisory review, management approval, etc.).
5. Records and information are maintained in accordance with laws and
regulations.
6. Assets and records are periodically verified.
7. Processes and procedures exist to support the continuity of information and
systems.
8. Development, implementation or changes to information systems are based on a
strategic plan for information systems and responsive to achieving organizational
strategic and operational objectives.
9. Implement IT solutions that align with the CRTC business environment, policy
goals, and statutory requirements.
Audit Objective 3
The effectiveness of monitoring activities (including performance monitoring
reports) and the extent to which the results of monitoring activities are used for
regulatory policy and management decision making.
Audit Criteria
1. External and internal environments are monitored to obtain information that may
signal a need to re-evaluate the organization’s objectives, policies and/or control
environment.
2. Monitoring of policy and program design options occurs in a regular and timely
manner.
3. Evaluation activities are used to identify policy and program strengths,
weaknesses and impacts (intended and unintended), as well as alternative ways of
designing policies, programs and initiatives.
4. Financial and non-financial reporting is reviewed and approved.
5. Appropriate and timely financial and non-financial reporting is communicated
internally and externally.
6. Management has identified appropriate performance measures linked to planned
results.
7. Management monitors actual performance against planned results and adjusts
course as needed.
CRTC Operational Audit 53
Audit Objective 4
An assessment of the effectiveness of on-going and planned initiatives that address:
a) the adequacy of information provided to regulatory stakeholders on the CRTC
website (i.e. to inform them of CRTC plans, processes, service delivery standards,
documentation requirements and results achievement): b) the CRTC actions taken
to address stakeholders comments received as part of industry surveys (e.g. dealing
with issues such as stakeholder satisfaction and recommendations for process
improvement): and c) the current processes of consultation with the CRTC
stakeholders in the setting and adjusting of the CRTC annual plans (e.g. is the
CRTC asking its stakeholders (industry, public) the right questions and is it
capturing the right information?).
Audit Criteria
1. Lines of communication exist between the organization, and stakeholders.
2. Suggestions, complaints and other input are captured and communicated to
relevant internal parties.
3. Input is sought, on a regular basis, from users and other stakeholders through
mechanisms such as environmental scanning, monitoring of public opinion and
client satisfaction.
4. Reaction to feedback from external stakeholders is managed in a coordinated
manner throughout the organization.
Audit Objective 5
An assessment of the effectiveness of interfaces by the four sectors, with other
sectors of the CRTC (e.g. legal, strategic communications and commission
members) in support of enhanced regulatory processes and results.
Audit Criteria
1. There is ongoing and transparent communication between the Operations
Committee (i.e. oversight body), management, Commissioners and the
Chairperson.
2. An information-sharing process exists to support the efficient and targeted
dissemination of relevant and reliable information to those that need it.
3. Open and effective channels exist for internal communications and feedback.
4. A clear and effective organizational structure is established and documented.
5. Memoranda of understanding, terms of reference or equivalent documents exist
for those government-wide or horizontal initiatives to which the organization
contributes.
Audit Objective 6
The effectiveness of CRTC management framework (e.g. practices and procedures
CRTC Operational Audit 54
relating to planning, organizing, controlling, leading, communications and
management of human and financial resources) within each of the sectors noted
above.
Audit Criteria
1. The CRTC has clearly defined and communicated strategic directions and
strategic objectives, aligned with its mandate.
2. The CRTC has in place operational plans and objectives aimed at achieving its
strategic objectives.
3. Human resource planning is aligned with strategic and business planning.
4. An HR Plan is documented and communicated and includes the following
elements:
• analysis of current and future resource and competency needs;
• analysis of key positions and succession planning; and
• training and development plan.
5. The CRTC identifies its current and future resource requirements and analyzes
them against existing human resource competencies and capacities.
6. The CRTC provides employees with the necessary training, tools, resources and
information to support the discharge of their responsibilities.
7. The CRTC has in place a system for the performance evaluation of employees.
8. Open and effective channels exist for internal communications and feedback.
9. The activities, schedules and resources needed to achieve objectives have been
integrated into the budget.
10. Financial management policies and authorities are established and
communicated.
11. Compliance with financial management laws, policies and authorities is
monitored regularly.
12. Management compares results achieved against expectations, on a periodic
basis.
13. Responsibilities and performance expectations to which managers and
supervisors are held accountable are formally defined and clearly communicated.
J ob descriptions and/or performance agreements should exist for this purpose and
be up-to-date.
CRTC Operational Audit 55
SUMMARY: RECOMMENDATIONS AND ANNEX 4
MANAGEMENTS RESPONSES
Recommendation #1
The Director of IM/IT should conduct a risk assessment on all CRTC key
databases and the databases considered to be at risk should be subject to a
Privacy Impact Assessment.
Risk Type Audit Risk Rating Impact
Compliance Moderate Potential that personal information could
be provided in a way that is not compliant
with the Privacy Act. Personal
information could be disclosed.
Response Office of
Primary
Interest
Timeline
A consultant will be hired to conduct a Threat and
Risk Assessment (TRA) on the DCS system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a Privacy
Impact Assessment (PIA) on the OWN system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a TRA on
the CRTC network.
Director
IM/IT
Summer
2010
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be carried
out on those priority systems revealed to be at
significant risk.
Director
IM/IT
Summer
2010
All remaining databases will undergo PTRAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary PIAs
(PPIA). Full PIAs will subsequently be carried out
on those priority systems revealed to be at
significant risk.
Director
IM/IT
October
2010
All remaining databases will undergo PPIAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
CRTC Operational Audit 56
Recommendation #2
The Director of IM/IT, in collaboration with senior management, should:
• Conduct a risk assessment to determine whether or not the CRTC is at
risk by not having an alternate data center processing site. If the risk
is deemed unacceptable, the CRTC should identify an appropriate
location to set up an alternate data processing site; and,
• Test the Business Continuity Plan and document the results.
Risk Type Audit Risk Rating Impact
Operations Minor Potential that CRTC could not provide
service to its clients and the public.
Response Office of
Primary
Interest
Timeline
IM/IT staff will test the current Business
Continuity Plan (BCP) to determine whether it
meets legislative and operational requirements.
Results and associated recommendations will be
presented to the Operations Committee.
Director,
IM/IT
Fall 2010
Recommendation #3
The Director of IM/IT should implement a rigorous monitoring and reporting
regime on the performance of the data centre.
Risk Type Audit Risk Rating Impact
Monitoring Moderate Potential process problems are not
identified for remedial action at an early
stage.
Response Office of
Primary
Interest
Timeline
A monitoring system called System Center
Operations Manager (SCOM) has been
implemented to record service information
related to the data center.
Director, IM/IT Spring 2009
Best practices for monitoring and reporting
regimes are currently being reviewed.
Director, IM/IT Spring 2010
Appropriate monthly monitoring and reporting
system to be implemented as required.
Director, IM/IT Fall 2010
CRTC Operational Audit 57
Recommendation #4
The Director of IM/IT should update the CRTC list of risks by conducting a Threat
and Risk Assessment of all services.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential security threats go undetected.
Response Office of Primary
Interest
Timeline
Director IM/IT The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be carried
out on those priority systems revealed to be at
significant risk.
Summer
2010
Director IM/IT All remaining databases will undergo PTRAs as
they are modified according to the existing
project life cycle. Business cases presented to
the IM/IT Steering Committee during the project
proposal cycle will include explicit reference to
the importance of carrying out these
assessments.
December
2010 and
ongoing
Recommendation #5
The Director of IM/IT should enhance the IM/IT Roadmap by identifying specific
projects, their estimated costs and how each project ties into the IT strategic
objectives, and the 3-Year Work plan.
Risk Type Audit Risk Rating Impact
Strategy Minor Potential that resources are used on
projects that do not further the objectives
of the CRTC.
Response Office of
Primary
Interest
Timeline
All suggested information will be incorporated
into the IM/IT Roadmap during the next project
proposal cycle, with projects for the next 3 years
clearly linked to the CRTC priorities and 3-Year
Work Plan through the use of business case
templates.
Director, IM/IT Summer
2010
CRTC Operational Audit 58
Recommendation #6
The Director of IM/IT should include in all project business cases the following
elements:
• Summary of alternatives considered;
• Resources required;
• Identification of risks for not approving the project;
• Identification of risks with mitigation strategies for the project;
• Reason(s) for selecting the recommended solution; and,
• Project management accountability framework.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential that not all pertinent information
is assessed before making the investment
decision.
Response Office of
Primary
Interest
Timeline
All future business cases presented to the IM/IT
Steering Committee will adhere to the existing
Business Case standards identified by the
auditors.
Director, IM/IT Summer
2010
Recommendation #7
The Executive Directors of Broadcasting and Telecommunications, in
collaboration with the Executive Director of the Policy Development and
Research, should ensure that the quarterly reports on service standards are
produced on a timely basis. Should reports be delayed, then the CRTC website
should provide a notice concerning the delayed reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential negative publicity regarding the
CRTC business processes.
Response Office of
Primary
Interest
Timeline
The CRTC has an obligation to report on an
annual basis, as set out in the Treasury Board
Policy on Service Standards for External Fees.
CRTC policies referring to quarterly reporting
have since been superseded by more recent
developments. While increased reporting
frequency is desirable, resource limitations have
led to the elimination of quarterly publication of
Executive
Directors
Ongoing
CRTC Operational Audit 59
service standard reports. The latest annual
reports include quarterly information and can be
found on the CRTC website. Regardless,
contracts for the automation of the reporting
process have been issued. Such automation may
permit a return to a quarterly reporting
frequency in the future.
Recommendation #8
The Director of Public Affairs should ensure that the telecommunication
coordinates are published on the CRTC website to allow easy access by the
stakeholders and that a logging system to catalogue calls and responses be
implemented.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential to reduce stakeholder
accessibility and the CRTC ability to
monitor and assess inquiries.
Response Office of
Primary
Interest
Timeline
Coordinates for the Single Point of Contact for
Small Telecom Service Providers have been
posted on the Telecommunications Sector page of
the CRTC website.
Director of
Public Affairs
December
2009
Work is currently underway to integrate the
operations of the Single Point of Contact for
Small Telecom Service Providers with the Rapids
case management system.
Director of
Public Affairs
April 2010
Recommendation #9
The Director of Public Affairs should develop a website page and or link(s) that
report on achievements, the status of stakeholders’ recommendations provided in
the surveys and reports, and a link to a calendar of future public appearances by
the Chairman, Commissioners and members of the senior management team, as
well as a link to key messages from management contained in various reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that the public is not fully aware
of the CRTC achievements.
Response Office of Timeline
CRTC Operational Audit 60
Primary
Interest
The CRTC website is our number one
communication tool. The diverse nature of
visitors to the CRTC website makes it quite
difficult to have a single webpage that would
contain all of the suggested information and be
relevant to all users. With the exception of a
calendar of future appearances, most of the
identified information can indeed be found in the
“Media” section of the CRTC website. All
speeches and news releases are posted to the
website and contain our key messages. CRTC
achievements are also communicated through the
Report on Plans and Priorities and the
Departmental Performance Report. Important
regulatory information, such as decisions and
information bulletins, has been organized by
industry and all documents can be accessed
easily and efficiently from a number of
redundant links. Finally, a dedicated “Consumer”
section exists for key information of particular
interest to the general public.
Director of
Public Affairs
Ongoing
A calendar of future appearances has not been
developed as the sensitive nature of the
Commission’s quasi-judicial role in the industry
can often necessitate last minute changes in
attendance due to a variety of potential conflicts.
The Director of Public Affairs will however
examine this proposal in the new fiscal year with
a view to providing as much information to the
public as possible.
Director of
Public Affairs
FY 2010/11
Recommendation #10
The Director General of Strategic Communications and Parliamentary Affairs, in
collaboration with the Secretary General and the Executive Directors of
Broadcasting, Telecommunications and PDR, should develop and implement an
integrated planning and reporting process and establish a pre-set forward agenda
to ensure that all major planning and reporting elements are reviewed at specified
times throughout the year.
Risk Type Audit Risk Rating Impact
CRTC Operational Audit 61
Operations Moderate Potential that senior management will not
maintain a strategic focus throughout the
year.
Response Office of Primary
Interest
Timeline
A CRTC Annual Corporate Planning and
Reporting cycle has been developed in
consultation with all sectors of the CRTC. It
will be presented for the approval of the
Operations Committee.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010
Development and approval of the cycle is the
first phase in establishing an integrated
planning and reporting process throughout the
organization. Additional processes will be
identified and incorporated into the process
over the next few months. (For example,
further input from members of the Corporate
Management Committee will be solicited in
J anuary 2010.)
Director General,
Strategic
Communications
and Parliamentary
Affairs
FY 2010/11
Recommendation #11
The Director General Finance and Administrative Services should continue to
reiterate to the Responsibility Centre Managers their responsibility for financial
budgeting and monitoring to ensure that they understand and provide complete,
accurate and timely financial information for the monthly budget summary report.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential that financial information
reported to senior management is not
complete and timely.
Response Office of Primary
Interest
Timeline
Procedures are in place whereby Finance staff
advises sector heads of the requirement to
review and update all their commitments
(Salary and O&M) on a monthly basis before
month end. In addition, as part of these
monthly notifications Finance staff have
advised sector heads that additional financial
training to sector management teams would
be provided upon request.
Director General,
Finance and
Administrative
Services
Ongoing
The requirement to do a comprehensive and
monthly review of all financial commitments
was recently re-emphasized by the Secretary
Secretary General November
2009
CRTC Operational Audit 62
General (Chief Financial Officer) to the
Executive Committee as part of the FY
2009/10 mid-year budget review.
Executive performance agreements have been
updated to reflect the expectation that all
members of the Executive Committee and
their management teams will review and
update their commitments monthly.
Director General,
Human Resources
Completed
Recommendation #12
The Champions, in collaboration with their project leads and the Director
General of Strategic Communication and Parliamentary Affairs, should ensure
that the Quarterly Reports on Future Direction are consistently published in a
complete and timely manner.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that incomplete reporting may
cause a decline in employee support for
the initiative.
Response Office of Primary
Interest
Timeframe
The Director General of Strategic
Communications and Parliamentary Affairs
will reiterate the importance of quarterly
reporting to the project leads and ensure that
all information is provided in a complete and
timely manner.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010 and
ongoing
Publication dates for quarterly reports will be
pre-set and the Strategic Communications
sector will ensure that all required information
is submitted in conformity with the relevant
timelines.
Director General,
Strategic
Communications
and Parliamentary
Affairs
April 2010
and ongoing
Recommendation #13
The Future Direction Champion and Project Manager for Streamlined Rules and
Regulations, in collaboration with the Executive Directors should:
• Identify the activities that constitute the Future Direction
streamlining and policy direction initiatives underway;
• Identify possible activities that can be integrated;
• Update the activity catalogue on a regular basis; and,
CRTC Operational Audit 63
CRTC Operational Audit 64
• Clearly establish responsibility for the management of the key area
as well as other initiative and ensure that complete and timely bi-
weekly progress reporting is implemented.
Risk Type Audit Risk Rating Impact
Operations
Risk
Minor Potential that not all activities are
effectively managed.
Response Office of
Primary
Interest
Timeframe
Priorities for the Streamlined Rules and
Regulations initiative were initially established
through a Process Mapping exercise that
identified process “hot spots” most in need of
attention. Remaining activities will continue to
be reviewed and re-prioritized on an ongoing
basis according to operational requirements and
resources.
Future Direction
Champion,
Streamlined
Rules and
Regulations
Ongoing
Project management will collaborate with
Strategic Communications to ensure that
quarterly status updates are issued on a regular
and timely basis.
Project
Manager,
Streamlined
Rules and
Regulations
J anuary 2010
and ongoing
Project scope and management responsibility
will be further clarified through the upcoming
development of Future Direction Exercise -
Phase II.
Future Direction
Champion,
Streamlined
Rules and
Regulations
April 2010
doc_832403287.pdf
The audit team found that there are effective and efficient regulatory processes, systems and procedures in support of achieving the Canadian Radio-television and Telecommunications Commission (CRTC) statutory mandate.
OPERATIONAL AUDIT REPORT
Findings, Recommendations and
Management Responses
May 2010
Catalogue No. BC92-74/2010E-PDF
ISBN #978-1-100-16159-4
TABLE OF CONTENTS
PREFACE 3
EXECUTIVE SUMMARY 4
INTRODUCTION
Background 6
Scope 6
Audit Objectives 7
Audit Methodology 8
FINDINGS, RECOMMENDATIONS AND MANAGEMENT RESPONSES
Regulatory Processes and Practices 9
Information Systems 13
Monitoring Activities 23
Interface with the Stakeholders 27
Interface between the Sectors 32
Management Control Framework 35
Annex 1 Risk Assessment Framework 49
Annex 2 List of Positions Interviewed 50
Annex 3 Audit Objectives and Audit Criteria 52
Annex 4 Summary: Recommendations
and Management Responses 56
CRTC Operational Audit 2
PREFACE
The operational audit of the CRTC is an initiative of its Chairman, the Honourable
Konrad W. von Finckenstein, Q.C. Shortly after his appointment, the Chairman requested
that an internal audit be undertaken in order to determine the efficiency and effectiveness
of the CRTC’s operations. It was his desire to identify which processes and systems are
working well, and which areas of the CRTC’s operations stood to benefit from increased
attention.
Funding for the operational audit was secured following the roundtable consultations
with CRTC fee stakeholders in 2007. These stakeholder consultations led to a two-year
temporary increase in the CRTC’s spending authority, which allowed the Commission to
undertake a review of its internal processes by outside audit professionals.
Professional audit services were contracted through the Public Works and Government
Services Canada Professional Audit Support Services (PASS) supply arrangement.
Independent oversight of the audit process was provided by the Office of the Comptroller
General’s Small Departments and Agencies Audit Committee (SDAAC). Pursuant to the
recommendation of the SDAAC, and in accordance with the Treasury Board of Canada
Secretariat’s Policy on Internal Audit, the following audit report was approved by the
CRTC Chairman on 25 May 2010.
CRTC Operational Audit 3
EXECUTIVE SUMMARY
Statement of Assurance
This operational audit engagement was planned and conducted in accordance with the
Treasury Board's Policy on Internal Audit and the Institute of Internal Auditors Standards
for the Professional Practice of Internal Auditing.
Audit Opinion
The audit team found that there are effective and efficient regulatory processes, systems
and procedures in support of achieving the Canadian Radio-television and
Telecommunications Commission (CRTC) statutory mandate. In addition, CRTC
initiatives being conducted to enhance regulatory processes, service delivery and
promoting management excellence were found to be effective. The audit evidence
gathered is sufficient to provide senior management with reasonable assurance of the
results derived from this audit.
Summary of Audit Findings
The audit team found that the regulatory processes, systems and procedures were
effective and efficient; that the services were delivered in a fair, open, transparent and
timely basis and in a manner that minimizes the regulatory burden on all stakeholders;
and that the staff was dedicated and prepared to do the work required, meeting the needs
of both industry and the public.
Through participation in the Future Direction initiative, CRTC staff have, in addition to
their regular duties, developed and implemented proposals that will enhance the
regulatory processes, service delivery and promote management excellence.
The existing information systems used to collect and provide information in support of
decision-making and accountability were found to be effective and meeting the needs of
the users. The CRTC has developed a strategic plan that identifies actions to be taken to
upgrade and improve the information systems to increase efficiency and reliability. The
plan has been initiated and is addressing a number of key Information Technology (IT)
priority areas.
Since the CRTC is a regulatory entity, it requires a variety of monitoring mechanisms,
both internal and external. These mechanisms were found to be effective and the results
were being regularly used for regulatory policy and management decision making.
Ongoing and planned initiatives to enhance stakeholder interaction with the CRTC were
found to be effective. A number of projects have been initiated to improve the
distribution and collection of information to and from stakeholders.
CRTC Operational Audit 4
The interface between the four sectors of Broadcasting, Telecommunications, Policy
Development and Research, and Corporate Services and Operations was found to be
effective in supporting the enhanced regulatory processes and results. An appropriate
governance structure was in place that ensured management direction, and plans and
actions were understood, appropriate and responsible.
The management control framework in place provides a clearly defined strategic
direction that is aligned to the CRTC mandate. The 3-Year Work Plan, the Report on
Plans and Priorities, and the Program Activity Architecture address the organization’s
outcomes, priorities and related action plans. A Performance Measurement Framework
was recently enhanced and will improve the ability of the CRTC to report performance in
future Departmental Performance Reports.
Summary of Audit Observations
The audit team identified four primary management areas that it believes, when
addressed, will enhance processes, systems and practices.
The project management authority for the key area of streamlined rules and regulations as
well as other initiative activities should be clarified and progress reporting should be
complete and timely.
An integrated planning and reporting process should be developed and implemented, and
a pre-set forward agenda established to ensure that all major planning and reporting
elements are reviewed at specified times throughout the year.
The responsibility for financial monitoring should be reiterated to Responsibility Center
Managers.
The Five Year IM/IT Roadmap should identify the specific projects for each of the five
years, the estimated project cost and how each project ties into the IT and corporate
strategic objectives.
Further elaboration on the above four management areas, as well as administrative areas
of potential improvement, are provided in the body of the audit report.
Management Action
The CRTC is addressing the recommendations in this report. Specific responses to each
recommendation are provided in the body of the audit report.
CRTC Operational Audit 5
INTRODUCTION
Background
The Canadian Radio-television and Telecommunications Commission (CRTC) was
established by Parliament in 1968. It is an independent public authority constituted under
the Canadian Radio-television and Telecommunications Commission Act and reports to
Parliament through the Minister of Canadian Heritage.
The CRTC is vested with the authority to regulate and supervise all aspects of the
Canadian broadcasting system, as well as to regulate telecommunications common
carriers and service providers that fall under federal jurisdiction. The CRTC derives its
regulatory authority over broadcasting from the Broadcasting Act. Its
telecommunications regulatory powers are derived from the Telecommunications Act and
the Bell Canada Act.
The CRTC serves the public interest by maintaining a balance between the cultural,
social and economic objectives of federal legislation on broadcasting and
telecommunications, taking into account the wants and needs of Canadian citizens,
industries and interest groups.
The CRTC fulfils its regulatory responsibilities by means of a number of inter-related
tasks, including:
• issuing, renewing and amending licences for broadcasting undertakings;
• holding public hearings on matters of significant public interest;
• making determinations on mergers, acquisitions and changes of ownership
in the broadcasting industry;
• approving tariffs and agreements for the telecommunications industry;
• fostering increased reliance on market forces for the provision of
telecommunications services and ensuring that regulation, where required,
is efficient and effective;
• monitoring competition and removing obstacles to competition;
• collaborating with industry to resolve competitive disputes;
• developing and implementing regulatory policies with a view to meeting
the objectives of the Broadcasting Act and the Telecommunications Act;
• monitoring, assessing and reviewing, where appropriate, regulatory
frameworks to meet policy objectives; and,
• monitoring the programming and financial obligations of broadcasting
undertakings to ensure compliance with regulations and conditions of
licence.
Scope
The operational audit included the review of the CRTC planning, regulatory and
reporting processes in the four sectors of Broadcasting, Telecommunications, Policy
CRTC Operational Audit 6
Development and Research and relevant sections within the Corporate Services and
Operations sector (i.e. Planning and Processes). It also included a review and assessment
of the on-going and planned initiatives to enhance regulatory processes, service delivery
and promote management excellence.
The CRTC budget allocation for fiscal 2009/10 is approximately 425 FTEs and a planned
spending of $46.0 M.
The audit was conducted at the CRTC headquarters in Gatineau, Quebec, during the
period April to September 2009. The audit focussed on the period 2008 to the present.
Audit Objectives
The overall objective was to review and assess: a) the effectiveness and efficiency of the
existing regulatory processes, systems and procedures in support of achieving the
CRTC’s statutory mandate; and, b) the CRTC initiatives (completed, on-going and
planned) being conducted to enhance regulatory processes, service delivery and promote
management excellence.
Specifically, the audit focused on the following six objectives:
1. The effectiveness and efficiency of regulatory processes and practices (e.g. public
hearing processes, application processes, scheduling, etc.) and to ensure that the
services are delivered in a fair, open, transparent and timely basis and in a manner
that minimizes the regulatory burden on all stakeholders. This effort included an
assessment of the on-going and planned CRTC initiatives to enhance the
regulatory processes and service delivery.
2. The effectiveness of existing information systems that are used to collect and
provide information in support of the CRTC regulatory processes (e.g. the
reliability of information systems for decision-making and accountability
purposes).
3. The effectiveness of monitoring activities (including performance monitoring
reports) and the extent to which the results of monitoring activities are used for
regulatory policy and management decision making.
4. An assessment of the effectiveness of on-going and planned initiatives that
address: a) the adequacy of information provided to regulatory stakeholders on
the CRTC website (i.e. to inform them of CRTC plans, process, service delivery
standards, documentation requirements and results achievement); b) the CRTC
actions taken to address stakeholders’ comments received as part of industry
surveys (e.g. dealing with issues such as stakeholder satisfaction and
recommendations for process improvement); and, c) the current processes of
consultation with the CRTC’s stakeholders in the setting and adjusting of the
CRTC Operational Audit 7
CRTC’s annual plans (e.g. is the CRTC asking its stakeholders the right questions
and is it capturing the right information?).
5. An assessment of the effectiveness of interfaces by the four sectors, with other
sectors of the CRTC (e.g. legal, strategic communications and commission
members) in support of enhanced regulatory processes and results.
6. The effectiveness of the CRTC management framework (e.g. practices and
procedures relating to planning, organizing, controlling, leading, communications
and management of human and financial resources) within each of the sectors
noted above.
Audit criteria were developed to assess each of these six objectives.
Audit Methodology
The audit was conducted in accordance with generally accepted auditing standards and
the requirements set out in the Treasury Board’s policy on Internal Audit. The audit
methodology respected the Office of the Auditor General’s methodology for Performance
Audits (formerly known as value-for money audits). The key tasks associated with the
audit approach involved reviewing CRTC documentation, systems and processes, and
interviewing CRTC staff.
To obtain an appreciation of the organization and its environment, management concerns
and insight into CRTC governance, risk management and controls, the audit team
completed a preliminary survey that included reviewing a number of documents and
interviewing members of the management team. The results of this review were used to
develop and discuss with management the audit program to use during the examination
phase of the audit. The Audit Team followed the submitted audit program.
CRTC Operational Audit 8
FINDINGS, RECOMMENDATIONS AND MANAGEMENT RESPONSES
Regulatory Processes and Practices
Conclusion
The audit team concluded that the CRTC has effective and efficient regulatory processes
and practices in place to ensure that the services are delivered in a fair, open, transparent
and timely basis as evidenced by our review of both the Planning and Process Directorate
procedures and the compilation of the briefing documentation prepared for the
Commission briefing books. Both processes ensured all submitted public comments and
interventions are considered during the decision-making process and the Commission
briefing books demonstrated that they contained a comprehensive summary of the
information required for decision-making. In addition, the audit team concluded that the
CRTC is actively pursuing initiatives that minimize the regulatory burden on
stakeholders and is addressing the profound change in the communications industry as a
result of technological and corporate convergence. This was demonstrated by our review
of the CRTC management framework over the many initiatives underway or completed
to address both the government’s policy direction to encourage increased reliance on
market forces in the telecommunication industry and the changing digital technologies
resulting in a convergence of the traditional industries.
Public hearing, scheduling and application processes
The Planning and Process Directorate oversees all public hearings. It is responsible for
completing tasks with respect to the scheduling, administration and logistics associated
with the hearing process. The unit is responsible for ensuring that all applications are
placed on the CRTC website for public awareness and comment, issuing the public notice
through the appropriate media outlets, forwarding the application to the appropriate
designated functional analyst and ensuring that interested parties have an opportunity to
file comments, interventions and replies. All negative interventions, counter
interventions and a summary of positive comments are assembled by the Directorate and
included in the briefing books. The audit team concludes that the Directorate is
following all applicable procedures, thereby ensuring all public comments and
interventions are considered during the decision-making process. This conclusion was
based on the interviews completed with the responsible managers, documentation review
of selected briefing books and review of the publicly accessible files on the CRTC
website.
The review of the briefing documentation prepared for the Commission briefing books
established that they contained a comprehensive summary of the information required for
decision making. In addition, interviews with responsible managers confirmed that the
Commissioners were satisfied with the level of support being provided. Our review of
the files indicated that service standards were consistently met resulting in no occurrences
of work not completed on time. Interviews with responsible managers established that
CRTC Operational Audit 9
CRTC Operational Audit 10
uncompleted work was an unacceptable option and that the staff would do whatever was
required to meet the deadline.
Increased workload has been generated by a significant increase in the number of public
hearings that are held each year
1
. In addition to a desire for greater transparency and
openness, this increase has been caused by certain administrative review processes now
being subject to the demands of a public hearing; a number of policy issues that have
become due for review; the change which is being experienced within the industry;
increased demand for new radio licenses; and the current state of the economy, which has
resulted in an increased frequency of ownership change.
The problem most frequently expressed by staff was the workload swings that were
necessitated by the scheduling of hearings. While there was some concern expressed at
the Executive Director level regarding the potential for over tasking of staff and staff
burn out, the majority of the managers interviewed expressed the opinion that they had
adequate resources to complete the assigned tasks. Research and policy analysis are not
the type of activities that would benefit from traditional work measurement assessment.
Determination of whether an adequate level of resourcing had been achieved must be
addressed by questions such as whether the work is being completed in an acceptable
manner within the required time frames and whether the associated resources appear to
be either underemployed or over worked.
The review of related documentation and discussions with staff revealed that:
• There is adequate subject-matter expertise for the conduct of policy research with
the majority being completed by CRTC staff with contracting out being utilized
when there is a requirement for specific expertise or third party advice;
• A process exists to solicit stakeholder comments related to service delivery and
significant change initiatives and management actions are communicated to the
appropriate stakeholders on a timely basis;
• CRTC issues a public notice that describes a proposed change and requests that
comments be received by a specific date when developing or changing a policy;
• Proceedings attract participation from opposing interests and public interest
groups and the degree of participation will vary depending on the subject matter.
The CRTC has established an administrative process that adequately addresses
and supports this participation;
• Established service standards were met as demonstrated by the file review;
• All comments and reply comments received from all stakeholders are made a part
of the public record of proceeding with material being treated as confidential and
excluded in limited circumstances when public disclosure could create
competitive disadvantage; and,
• All decisions and associated rationale are posted on the CRTC website.
1
Public Hearing statistics maintained by the Planning and Process Directorate indicated the following:
2005/06 – 13 hearings, 27 days;
2006/07 – 16 hearings, 46 days;
2007/08 – 20 hearings, 60 days; and,
2008/09 – 18 hearings, 66 days.
Initiatives to enhance the regulatory processes and service delivery
In recent years there have been two significant influences that have required theCRTC to
review the delivery of its mandate. The government issued a policy direction in
December 2006 on thederegulation of the Canadian telecommunication industry and is
placing an increased emphasis on market forces. In addition, the communications
industry is undergoing a profound change as new digital technologies transform how
Canadians communicate, how the CRTC delivers its mandate and how the public is
informed and entertained. The broadcasting and telecommunications industries are
embracing new technologies and moving to a convergence of their traditional industries.
The audit team reviewed the management framework that was in place to address the
CRTC response to these influences. In 2008, an internal task force was created to
develop streamlined rules and regulations. The task force developed terms of reference
articulating the objective, scope, background and team membership. The project
manager’s initial action was to identify and document all activities generated by both the
policy direction and the Future Direction initiative. The audit team reviewed the
resulting activity catalogue with the responsible managers and found it to be an effective
management tool, except for concerns raised by the audit team under the Management
Control Framework section of this report.
The CRTC has implemented a number of initiatives to minimize the regulatory burden in
the industry and streamline its regulatory processes. Some examples are as follows:
• Under the realigned structure, activities common to both broadcasting and
telecommunications were undertaken by the Policy, Development and Research
Sector. PDR has been tasked to focus on convergence policy, social policy and
arbitration. These activities will become more important in an increasingly
deregulated industry.
• The CRTC’s regulatory policies were reviewed and provided auditors with
sufficient evidence showing how the CRTC eliminated certain data reporting
requirements.
• In telecommunications, process improvements have been ongoing as a result of
the policy direction and other initiatives implemented during the last few years.
An example is the implementation of streamlined processing (10 day approval) to
assess requests for retail tariff modifications.
• The Broadcasting and Telecommunications Rules of Procedures have been
combined into a single set of rules that incorporates input from CRTC senior staff
and is currently under review at the Department of J ustice. It was expected to be
ready for publication and implementation during fiscal year 2009/10, however
implementation is now forecast for fiscal year 2010/11.
CRTC Operational Audit 11
• The CRTC is currently in the process of reviewing the procedures in the Public
Hearing process and has identified opportunities for process development.
Process maps of the three major public hearing processes were developed which
reveal “hot spots” where process improvements could be made.
• The CRTC has expanded the scope of the exemption order respecting radio and
television temporary network special event type 1 undertakings to permit these
undertakings to distribute programming that is of a special and recurring nature
(i.e. CRTC exemption order 2009-18) resulting in less filing by the industry.
• The CRTC had discussions with stakeholders which resulted in the
implementation of streamlining measures pertaining to the filing of channel line-
ups on renewal, application forms, replacing paper services area maps with maps
in electronic format and eliminating financial information from the application
form for new BDUs (i.e. CRTC Info Bulletin 2009-384).
• Streamlining the process for transactions involving the transfer of shares (i.e.
Information Bulletin Broadcasting Circular CRTC 2008-8).
• Internal processes have been put in place for transactions involving the
acquisition of assets and certain transfers of shares to ensure that a draft decision
is distributed on time to the Panel and the Full Time Members (FTM) for
consultation to allow publication of the decisions within 35 days from the first
day of the public hearing.
• Annual no change confirmation or updating versus a complete package of
ownership information accompanying licence renewal applications.
• All licences under the same ownership being renewed as a group at the same time.
• The CRTC introduced Telecom Regulatory Policy 2009-183, which eliminated
certain data reporting requirements such as reporting on pay-telephone
competition and tracking customer complaints regarding modem hijackings. In
addition, telecommunications service providers are no longer required to file an
annual report on the affordability of telephone services. The CRTC will continue
to gather this information from other sources.
• Broadcasting Circular CRTC 2005-466 allowed applicants to file their
applications and all related documents in electronic form using ePass. Using
ePass eliminates the requirement to file a hard copy of the documents. This
expedited the filing process by removing the need for the CRTC to verify
electronic and hard copy versions of an application.
• An MOU was developed between the CRTC and Statistics Canada concerning the
collection and sharing of information for telecommunications and broadcasting
surveys. This arrangement was intended to ensure that the information released
CRTC Operational Audit 12
by the Parties is as consistent as possible, that the Parties do not duplicate efforts,
and that no undue burden is imposed on respondents.
• The CRTC updated the look and feel of its website in February 2009.
The audit team concluded that theCRTC is actively engaged with its stakeholders to
minimize the regulatory burden in the industry and streamline its regulatory processes.
This was demonstrated by the many initiatives completed and implemented, and the
management framework that theCRTC has in place to manage the initiatives still
underway.
Information Systems
Conclusion
The audit team concluded that the Information Systems used to collect and provide
information in support of the CRTC regulatory processes are effective in providing
reliable information for decision making and accountability. This was demonstrated by
the CRTC website that provides the public and staff access to information and
applications. The use of pre-set electronic forms requiring specific data to be input into
fields minimizes the risk of inaccurate data. In addition, analysts review key information
to further ensure its accuracy. The audit team noted that the CRTC has policies in place
to ensure information is compliant with Privacy Legislation; however, the CRTC has
completed only two privacy assessments. The CRTC should conduct a risk assessment
on all CRTC key databases, and the databases considered to be at risk should be subject
to a Privacy Impact Assessment. Although the CRTC has adequate back up systems to
ensure continuity of information and systems, it does not have an alternate data
processing site. The CRTC should assess whether or not it believes this is an acceptable
risk and if determined to be unacceptable, the CRTC should identify a location for an
alternate data processing site. The CRTC conducted a Threat and Risk Assessment in
2001, which should be updated. The audit team noted that IM/IT strategic planning was
adequate; however, it did not identify the estimated costs of specific projects and how
each project was integrated into the IT strategic objectives and the CRTC 3-Year Work
Plan.
This conclusion is based on the audit team’s review of the CRTC policies related to data
and information. Auditors conducted reviews to provide reasonable assurance that the
CRTC complied with Privacy Legislation. The CRTC website was used by the auditors
to ensure it provided easy access to decisions, information and services. The audit team
observed data entry into certain systems and at the same time observed information
displayed on the screens of these systems to obtain reasonable assurance of information
accuracy. It observed demonstrations of the key functions within the Ownership
Structure of the Industry (OWN), Application Support System (APP), Master Address
Database (MAD), Data Collection System (DCS), Radio Assessment of Industry (RAP)
and RAPIDS applications. IM/IT plans were reviewed to determine whether these plans
CRTC Operational Audit 13
were linked to the CRTC business plans and IM/IT business cases provided some
information concerning changes to systems.
User service and access
The CRTC website provides easy access to decisions, information and services as well as
the submission of applications, interventions and comments. The website, combined with
the CRTC Intranet site (“the Zone”), provides both the public and internal CRTC users
access to information and applications. For example, the Ownership system allows
analysts to access the system for both entering data and making inquiries. Personnel in
other areas have access for inquiries, using a more extensive inquiry facility. An
interface with the APP system provides OWN with updated identification information on
a periodic basis.
Accuracy of transaction coding and processing
Input of transactions to the various components of the applications is accomplished
through pre-set electronic forms, which require specific data to be input into fields. Since
the information is provided electronically, it is very difficult to obtain the original direct
source data as evidence to ensure data accuracy. However, the procedures observed
during the audit provided assurance that the data is accurate. For instance, analysts
review key information (e.g. applications, surveys, forms) to ensure accuracy.
The CRTC introduced its web-based Data Collection System (DCS) in J anuary 2004.
DCS employs a secure, encrypted connection between the entity submitting data and the
CRTC thereby improving the accuracy, ease and timeliness of data submission.
External clients using ePass expedite the data filing process by removing the need for the
CRTC to verify both electronic and hard copy versions of an Application. Data
submitted directly by external clients without being input by CRTC staff is as accurate as
possible. The CRTC informed the audit team that some stakeholders had experienced
difficulty when submitting very large application files using ePass, since PWGSC has
restricted and reduced the size of the files being downloaded. The CRTC has reviewed
the comments and problems reported by stakeholders and will be improving the ePass
process through the implementation of a new software security solution. This will result
in a more effective and efficient electronic application system to meet stakeholder
demands. In the meantime, the stakeholders can utilize the CRTC central point of contact
to address their inquiries.
Records/information maintained in accordance with laws and regulations
The CRTC has several policies related to records, data and information that provide
assurance that the records, data and information are in compliance with Privacy
Legislation. These policies include the CRTC Policy on the Identification and Release of
Information, Guidelines on the Release and Protection of Information, Guide to
Information Protection and Destruction, and the CRTC Directive on Email Management.
CRTC Operational Audit 14
All databases are secured using a user-id/password and managers must approve each
employee’s access to applications using each database. User profiles restrict access to
read, write or update information/data on the database.
Multiple firewalls are in place. Intrusion detection and protection are set up to monitor
both internal and external traffic.
The CRTC subscribes to a software service that allows the CRTC to remotely wipe out a
laptop hard disk of any web-connected laptop reported to be missing.
The CRTC is responsible for demonstrating that its collection, use and disclosure of
personal information respect the Privacy Act and privacy principles throughout the
initiation, analysis, design, development, implementation and post-implementation
review phases of their program and service delivery activities. This is accomplished
through the Privacy Impact Assessment (PIA) Policy.
The PIA Policy is one of several tools designed to assist Canadians in understanding how
the government handles their personal information and to trust it to do so responsibly.
The policy is based on privacy principles common to all data protection regimes. These
principles are enumerated in the "Code of Fair Information Practices" in the federal
Privacy Act as well as in the ten privacy principles attached to the Personal Information
Protection and Electronic Documents Act.
The PIA Policy states that Institutions must identify all personal information associated
with business processes. This should be accomplished by developing a detailed
description and analysis of the data flows. Key components include the business process
diagrams, data flow tables and system and infrastructure architectures. The Policy states
that departments and agencies must ensure and document that privacy principles,
legislation and policies are adhered to. The CRTC does this through its Important
Notices on the website.
The PIA Policy does state that institutions must document their evaluation of privacy
risks, the implications of those risks and their discussions of possible remedies, options
and recommendations to avoid or mitigate such risks.
The end result of the PIA Policy is assurance that all privacy issues have been identified
and resolved or mitigated. The associated documentation forms the basis for seeking the
advice of and notifying the Privacy Commissioner as well as for assuring the public that
privacy issues have been addressed.
The CRTC has completed only two PIAs. One was on the Do Not Call List (DNCL) and
a second was on the Data Collection System (DCS). In 2004 a preliminary PIA was
completed on the APP system and was intended to be fully completed prior to proceeding
with the implementation of the ePass initiative. However, the complete PIA for the APP
system has not yet been completed.
CRTC Operational Audit 15
A PIA is required when there is new data matching due to changes in the business
procedures and/or systems. In absence of a PIA completed on other databases, the CRTC
is at risk of not complying with the "Code of Fair Information Practices" in the federal
Privacy Act or the ten privacy principles attached to the Personal Information Protection
and Electronic Documents Act. Without a robust PIA process, the CRTC is at risk of
disclosing personal information without appropriate authorization.
The CRTC has a Privacy Statement on its website, although there have been only two
PIAs completed. The Privacy Statement indicates that the CRTC does not automatically
gather personal information and that personal information is only gathered if it is
supplied by clients and/or the public. The Privacy Statement also states that “The CRTC
will collect, use and disclose personal information in accordance with the Privacy Act”.
In order to demonstrate compliance with the above statement the CRTC requires a more
robust PIA process that ensures all its databases are at least assessed as to the risk that
personal information could be provided in a way that is not compliant with the Privacy
Act. The audit team did not find any examples of information being disclosed
inappropriately.
Recommendation:
The Director of IM/IT should conduct a risk assessment on all CRTC key
databases and the databases considered to be at risk should be subject to a
Privacy Impact Assessment.
Risk Type Audit Risk Rating Impact
Compliance Moderate Potential that personal information could
be provided in a way that is not compliant
with the Privacy Act. Personal
information could be disclosed.
Management Response:
Response Office of
Primary
Interest
Timeline
A consultant will be hired to conduct a Threat and
Risk Assessment (TRA) on the DCS system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a Privacy
Impact Analysis (PIA) on the OWN system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a TRA on
the CRTC network.
Director
IM/IT
Summer
2010
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be carried
out on those priority systems revealed to be at
Director
IM/IT
Summer
2010
CRTC Operational Audit 16
significant risk.
All remaining databases will undergo PTRAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary PIAs
(PPIA). Full PIAs will subsequently be carried out
on those priority systems revealed to be at
significant risk.
Director
IM/IT
October
2010
All remaining databases will undergo PPIAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
Continuity of information and systems
Databases and Data Stores are backed up at regular intervals and backup media are sent
to the National Archives.
The Treasury Board Secretariat in the TBS Business Continuity Plan (BCP) standard
identifies four elements of a business continuity program as:
• The establishment of a BCP governance structure;
• The conduct of a business impact analysis;
• The development of business continuity plans and arrangements; and,
• The maintenance of BCP program readiness.
The audit team reviewed the CRTC Business Continuity Plan Program Guide and found
that the TB Standards had been met, with the exception of training and testing of the plan.
The audit team found no evidence that the training had been provided or that the Business
Continuity Plan had been tested. The audit team also noted that there is no data centre
alternate processing site in the Continuity Plan. Without an alternate processing site, the
CRTC is at risk of not being able to provide service to its clients and the public. Clients
would not be able to submit data, hearing dates would be unavailable and notices would
not be published. Although there is no government standard regarding an alternate data
center processing site, it is considered a best practice. CRTC management should be
aware of all the risks associated with not having an alternate data center processing site.
CRTC Operational Audit 17
Recommendation:
The Director of IM/IT, in collaboration with senior management, should:
• Conduct a risk assessment to determine whether or not the CRTC is
at risk by not having an alternate data center processing site. If the
risk is deemed unacceptable, the CRTC should identify an
appropriate location to set up an alternate data processing site; and,
• Test the Business Continuity Plan and document the results.
Risk Type Audit Risk Rating Impact
Operations Minor Potential that CRTC could not provide
service to its clients and the public.
Management Response:
Response Office of
Primary
Interest
Timeline
IM/IT staff will test the current Business
Continuity Plan (BCP) to determine whether it
meets legislative and operational requirements.
Results and associated recommendations will be
presented to the Operations Committee.
Director,
IM/IT
Fall 2010
Data center monitoring
The audit team noted that there were no monitoring reports on the performance of the
CRTC data center. Typical monitoring and reporting activities would include the amount
of “up time”, CPU utilization, disk utilization and network utilization. The CRTC is at
risk of not being able to identify, at an early stage, potential processing problems. A
rigorous monitoring and reporting regime on the performance of the data center would
mitigate the risk.
Recommendation:
The Director of IM/IT should implement a rigorous monitoring and reporting
regime on the performance of the data centre.
Risk Type Audit Risk Rating Impact
Monitoring Moderate Potential process problems are not
identified for remedial action at an early
stage.
CRTC Operational Audit 18
Management Response:
Response Office of
Primary
Interest
Timeline
A monitoring system called System Center
Operations Manager (SCOM) has been
implemented to record service information
related to the data center.
Director, IM/IT Spring 2009
Best practices for monitoring and reporting
regimes are currently being reviewed.
Director, IM/IT Spring 2010
Appropriate monthly monitoring and reporting
system to be implemented as required.
Director, IM/IT Fall 2010
Threat and risk assessment
The last Threat and Risk Assessment (TRA) was conducted in March 2001 where the
threats and risks were deemed Medium to Low. Since the TRA aids in the determination
of security requirements, the CRTC should conduct a TRA for every program, system or
service. For example, a TRA should be conducted for physical security of the data
centre; a TRA on Spyware/Viruses; a TRA on the electronic transmission of data; a TRA
on the Application Development Process on access controls; and, a TRA on some of the
larger systems such as APPS, DCS, Finance, electronic hearings, etc. The TRA will
assist the CRTC to answer the following key questions:
• What needs to be protected?
• Who/What are the threats and vulnerabilities?
• What are the implications if data were damaged or lost?
• What is the value of data to the organization?
• What can be done to minimize exposure to data loss or damage?
Threat and Risk Assessments can be short and simple or detailed and rigorous, depending
on the sensitivity, criticality and complexity of the program, system or service being
assessed.
Recommendation:
The Director of IM/IT should update the CRTC list of risks by conducting a
Threat and Risk Assessment of all services.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential security threats go undetected.
CRTC Operational Audit 19
Management Response:
Response Office of Primary
Interest
Timeline
Director IM/IT The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be
carried out on those priority systems revealed
to be at significant risk.
Summer
2010
Director IM/IT All remaining databases will undergo PTRAs
as they are modified according to the existing
project life cycle. Business cases presented to
the IM/IT Steering Committee during the
project proposal cycle will include explicit
reference to the importance of carrying out
these assessments.
December
2010 and
ongoing
Strategic planning for information systems
The IM/IT Roadmap presented in J une 2009 is an example of a good planning document.
It identifies the actions to be taken to upgrade the IM/IT environment for increased
efficiency, reliability, mobility and around-the-clock use from CRTC workers. It
provides a five year course of action based on principles generally found in the CRTC 3-
Year Work Plan such as “establishing modern and secure infrastructure to meet
stakeholder demands”.
Some of the projects discussed in the roadmap include:
• Revamp the CRTC web presence and migrate to a Web 2.0 environment;
• Develop an approach for CRTC content management and web publishing;
• Review the current tools, processes and procedures used in exchanging
information between the CRTC, the industry and the public; and,
• Develop a Business IM/IT Partnership Charter spelling out the principles,
mechanisms and approaches to be used in establishing and operating the
partnership.
The five year IM/IT Roadmap, as discussed with the IM/IT Steering Committee in March
2009, was developed using a discussion guide that described the scope of the roadmap as
a way for the CRTC to move forward with respect to:
• Enhancing the alignment between its business lines and IM and IT;
• Enabling business lines to extract maximum business value from their IM and IT
investments; and,
• Creating appropriate success conditions for alignment and IM and IT business
contribution.
CRTC Operational Audit 20
Although the IM/IT Roadmap was a good starting point, it did not identify the specific
projects for each of the five years, and it did not identify project estimated costs or how
each project ties into the IT Strategic Objectives or the six coordinated actions.
Estimated costs and linkages to the IT strategic objectives or six coordinated actions were
not found in any other document provided to the audit team. All the information should
be in one document facilitating management retrieval.
For example, the IM/IT Proposed Projects for 2009/10 (May 24, 2009) lists the IM/IT
accomplishments for 2008/09 and lists the carry over projects for 2009/10; however, it
does not show how the completed projects or the carry over projects link to the six
coordinated actions of:
• Infrastructure enhancement;
• Rationalization of systems, tools and processes;
• Enhanced stakeholder interfaces;
• Effective Information and Knowledge Management;
• Effective Partnership; and,
• Systemic Organizational Innovation.
The IM/IT Roadmap does not show how the IM/IT projects are linked to the three
Commission priorities found in the 3-Year Work plan for 2008 to 2011 of:
• A more focused regulatory approach;
• Greater outreach to stakeholders; and,
• An improved organization.
Although there is no documentation to show the direct linkage between the IM/IT
Roadmap and Business Cases to the 3-Year Work Plan and RPP, the audit team was able
to link some of the projects. For example:
• The Upgrade to Ownership application is based on the 3-Year Work Plan (year
2008/09) for implementing a broadcasting ownership reporting mechanism; and,
• SQL2005 upgrade, SAN upgrade, Vista Upgrade, and other software
implementation are all based on the 3-Year Work Plan to establish modern and
secure infrastructure to meet stakeholder demands.
An IM/IT Steering Committee was created for approving IM/IT projects and ensuring
that the CRTC has policies, technologies and the infrastructure to conduct business
electronically with all stakeholders. Members of the Committee bring forward priorities,
issues and problems. The Committee consists of members from IM/IT, Broadcasting,
Telecommunications, Media/Policy Development and Research, Communication, Legal
and Finance. It meets three times a year.
CRTC Operational Audit 21
Although this process is used to approve projects, there is no documentation showing
how each approved project ties into the IT Strategic Objectives, the six coordinated
actions or to the 3-Year Work Plan.
Without linking the IM/IT projects to the IM/IT Strategy or to the 3-Year Work Plan,
there is a risk that resources will be used on projects that do not further the objectives of
the CRTC.
Recommendation:
The Director of IM/IT should enhance the IM/IT Roadmap by identifying specific
projects, their estimated costs and how each project ties into the IT strategic
objectives, and the 3-Year Work Plan.
Risk Type Audit Risk Rating Impact
Strategy Minor Potential that resources are used on
projects that do not further the objectives
of the CRTC.
Management Response:
Response Office of
Primary
Interest
Timeline
All suggested information will be incorporated
into the IM/IT Roadmap during the next
project proposal cycle, with projects for the
next 3 years clearly linked to the CRTC
priorities and 3-Year Work Plan through the
use of business case templates.
Director, IM/IT Summer
2010
Project business case
A project business case provides a detailed investment proposal, including an analysis of
the costs, benefits and risks associated with a proposed investment and offers reasonable
alternatives. It provides information necessary to make a decision about whether or not a
project should proceed. It is the indispensable first activity in the lifecycle of an IT
investment.
The audit team reviewed eight project business cases and found that they are incomplete,
do not define accountability and do not adhere to IT Business Case standards.
Specifically, business cases lack the following:
• Benefits of the solution are not quantified;
• There is no project sponsor;
CRTC Operational Audit 22
• There is no project manager;
• There is no discussion of the different options studied to solve the business
problems;
• There are no timing, resources required or cost estimates; and,
• There is minimal risk identification.
Recommendation:
The Director of IM/IT should include in all project business cases the following
elements:
• Summary of alternatives considered;
• Resources required;
• Identification of risks for not approving the project;
• Identification of risks with mitigation strategies for the project;
• Reason(s) for selecting the recommended solution; and,
• Project management accountability framework.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential that not all pertinent information
is assessed before making the investment
decision.
Management Response:
Response Office of
Primary
Interest
Timeline
All future business cases presented to the
IM/IT Steering Committee will adhere to the
existing Business Case standards identified by
the auditors.
Director, IM/IT Summer
2010
Monitoring Activities
Conclusion
The audit team concludes that overall the monitoring activities are effective and are used
for regulatory policy and management decision making. This was demonstrated by the
audit team’s review of the current suite of monitoring reports available to the CRTC staff.
All managers interviewed commented favourably that the systems providing the reports
were constantly being updated as a result of both internal and external requests for system
updates.
This conclusion is based on conducting interviews and observing how certain IT systems
collect and report on information. In addition, the audit team noted the quality assurance
CRTC Operational Audit 23
practices of data submitted by external clients as performed by CRTC analysts. Various
monitoring reports were reviewed along with information obtained from the CRTC
website.
Regulatory monitoring
The Communication Monitoring Report was reviewed to determine what information was
being monitored and reported. The report was also used to identify which IT systems
were used to collect and report data.
There are several systems that collect information used by the CRTC to facilitate the
monitoring of regulatory policy. These systems include:
• The Ownership Structure of the Industry (OWN) - This system maintains
information on the ownership and control structure of companies and individuals
involved in the Canadian Broadcasting industry. It assists the CRTC to determine
the entity’s eligibility to hold a licence, the extent of Canadian control and all
entities involved in the control chain;
• The Application Support System (APP) – This system is used to register
applications filed by the Broadcasting and Telecommunications industries. It
provides the ability to disseminate the application to the processing team,
schedule the process, monitor the progress and report activities and statistics for
all broadcasting and telecommunications applications, decisions, licences and
undertakings. It also provides the calendar of activities for the CRTC, APP
Registers, schedules and tracks reports, and produces statistics for all
Broadcasting and Telecom applications, licences, undertakings, decisions and the
CRTC Calendar of Activities;
• The Radio Assessment of Industry (RAP) – This system is a tool for analysts to
assess logger tapes submitted by radio stations for the purpose of monitoring
compliance to the regulations and to allow for the collection and analysis of
popular music data; and,
• The Data Collection System (DCS) – This system is a web based system that
archives, reports, and analyzes data related to competition in Canadian
telecommunications markets. The system supports the CRTC activities relating to
preparing the annual Report to the Governor in Council (GIC): Status of
Competition in Canadian Telecommunications Markets -
Deployment/Accessibility of Advanced Telecommunications Infrastructure and
Services; produces a Telecommunications registration list membership;
international telecommunications licence renewals; telecommunications fees; and,
the Contribution Collection Mechanism (CCM). The system also contains contact
details of individuals and entities needed to administer and collect the previously
mentioned data, pursuant to Telecom Circulars 2003-1 and 2005-4. These
Circulars describe the annual collection of telecom industry data using DCS and
also describe administrative changes in the annual collection of telecom industry
data that the Commission implemented to better coordinate and streamline a
number of activities it undertakes to regulate and monitor the Canadian
CRTC Operational Audit 24
telecommunications industry. The annual data collection is also used to support
CRTC activities relating to maintaining and updating telecom entity registration
lists, international licences and telecom fees.
Information from these systems is used by the CRTC staff responsible for regulatory
monitoring and policy decision making. These systems are constantly being upgraded to
accommodate requests from internal and external sources in order to make processing of
applications and data more efficient.
Monitoring report review and approval process
While conducting the audit fieldwork, it was noted that reports are reviewed by Analysts
and Managers before they are actually distributed or posted onto the CRTC website. The
CRTC Web Publishing Guide states that all web content changes or additions must be
signed off by the designated content approver (i.e. Director or Director General
depending on content) before they can be submitted for web publishing.
Another example relates to the decision process used to approve the publishing of the list
of radio stations tested for compliance with Canadian content requirements. After the
analyst creates the report from the data in the RAP system, the report is sent to the
manager for final approval before posting the report to the CRTC website.
The Data Collection System (DCS) is used by clients for input of data for related
statistical reporting/collection. The data is input using pre-set electronic forms. After the
data is input by the clients the responsible analysts perform a quality assurance review on
the data. Once the data is deemed to be reasonable, the analysts notify clients to formally
submit the data into the system.
After analyzing radio programming information sent by radio stations, analysts produce a
report called the “Analysis Report”. The report is approved by management prior to
distribution.
Other external and internal monitoring
The audit team reviewed the list of different reports that relate to policy and program
design options. They found that there are a large number of reports produced on a
regular basis.
On its website, the CRTC publishes a Summary of Local Radio Programming, which
identifies, for those radio stations selected, the amount of local programming provided by
the stations. The staff reviewing the radio content reports stated the reports were meeting
the CRTC responsibility to monitor whether or not stations were meeting Canadian
content requirements.
The CRTC Broadcasting Policy and Monitoring Report and the Telecommunications
Monitoring Report were combined into the Communications Monitoring Report. This
CRTC Operational Audit 25
report provides a more holistic view of the industries and their markets. The report
expands on the performance indicators and trends reported in previous broadcasting and
telecommunications monitoring reports. The audit team concluded, based on interviews
with managers and a review of the report contents, that it meets their needs.
The Web Trends report published internally monitors how the CRTC website is used. It
identifies the number of users visiting the CRTC website, average visits per visitor,
average visitors per day and average visit duration.
A website broken links report is produced on a weekly basis by the Strategic
Communications and Parliamentary Affairs sector. There were approximately 80,000
broken links before the Broken Link Report was produced. A little over one year of
using the report has resulted in a significant reduction in broken links.
The CRTC produces a quarterly report for broadcasting applications in accordance with
Broadcasting Circulars CRTC 2006-1 and 2006-2. Both of these circulars indicate that
the CRTC will post quarterly statistics on its website to allow evaluation of its
performance in meeting the service standards. Auditors observed that the quarterly
reports for April 1, 2008 to March 31, 2009 were published on the CRTC website in
September 2009. Since the quarterly reports on broadcasting service standards are not
regularly published in a timely manner, the audit team concludes that they are not being
used effectively because the information is dated. Although the CRTC has not published
this information on a quarterly basis, the audit team found that the CRTC meets its
obligations to report to the TBS on service standards for external fees annually through
the Departmental Performance Report.
Telecommunications service standards and performance measures for Type 1 and 2 and
local forbearance Part VII applications are currently monitored pursuant to Telecom
Circular CRTC 2006-11, and Telecom Decision CRTC 2006-15, as amended by Order in
Council P.C. 2007-532, 4 April 2007, respectively. The service standards and
performance measures for (a) tariff applications and intercarrier agreements, and (b)
applications regarding the destandardization and/or withdrawal of a service processed
during the fiscal year are also monitored pursuant to Telecom Circular 2006-11.
The Telecommunications Branch has worked with the CRTC IM/IT team in recent
months to develop systems and tools to allow (1) for the efficient tracking of performance
related to the processing of all applications received from the industry, and (2) for the
production of timely reports at the end of each reporting period. As of September 2009,
the CRTC website sets out statistics related to its Telecommunications service standards
and performance measures in a single report, with the statistics being presented in a clear,
concise, and user-friendly format. The report allows the public to monitor the ability of
the CRTC to deal with all applications received in a timely manner. The system tools
and the report allow CRTC management to track and analyse its performance with
respect to the processing of the different types of applications.
The CRTC has complied with Telecom Circular CRTC 2006-11 by posting service
standard results on the CRTC website following the March 31, fiscal year-end. However,
CRTC Operational Audit 26
the CRTC also indicated in Telecom Circular CRTC 2006-11 that it expected to post
service standards on a quarterly basis for information purposes. Posting quarterly reports
on the CRTC website would improve the effectiveness of information provided to
stakeholders and harmonize this reporting function with that of the Broadcasting Sector.
Recommendation:
The Executive Directors of Broadcasting and Telecommunications, in
collaboration with the Executive Director of the Policy Development and
Research, should ensure that the quarterly reports on service standards are
produced on a timely basis. Should reports be delayed, then the CRTC website
should provide a notice concerning the delayed reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential negative publicity regarding the
CRTC business processes.
Management Response:
Response Office of
Primary
Interest
Timeline
The CRTC has an obligation to report on an
annual basis, as set out in the Treasury Board
Policy on Service Standards for External Fees.
CRTC policies referring to quarterly reporting
have since been superseded by more recent
developments. While increased reporting
frequency is desirable, resource limitations
have led to the elimination of quarterly
publication of service standard reports. The
latest annual reports include quarterly
information and can be found on the CRTC
website. Regardless, contracts for the
automation of the reporting process have been
issued. Such automation may permit a return to
a quarterly reporting frequency in the future.
Executive
Directors
Ongoing
Interface with the Stakeholders
Conclusion
Generally, the audit team concluded that the information the CRTC provides to its
regulatory stakeholders is adequate; the CRTC is actively addressing stakeholders’
CRTC Operational Audit 27
comments; and the CRTC has an effective stakeholder consultation process to set and
adjust annual plans.
Website information provided to stakeholders
The CRTC website provides complete and timely information to regulatory stakeholders
with respect to decisions, notices, orders and information bulletins as well as information
on CRTC annual pans and priorities. It also provides the contact coordinates (i.e.
telephone number/e-mail address) to reach Broadcasting and Client Services
representatives with questions, suggestions and complaints.
The audit team observed that the telecommunications single point of contact, put in place
in May 2009, does not appear on the website. The Broadcasting and Client Services
central points of contact are displayed on the website and provide easy access for the
stakeholders to contact the CRTC. The Telecommunications stakeholders who do not
know the Telecommunications single point of contact number can contact the numbers
displayed on the website and their calls are redirected to the appropriate person. Since
the CRTC has provided the coordinates on its website for the other two functions it
would be beneficial to provide the stakeholders with direct access to the
telecommunication function.
CRTC has put in place an internal data base tracking system (i.e. Rapids), to manage
enquiries, complaints or suggestions related to Broadcasting, Telecommunications and
Client Services enquiries, but is only tracking the Broadcasting and Client Services
enquiries. The system is not used to track the telecommunications enquiries.
Recommendation:
The Director of Public Affairs should ensure that the telecommunication
coordinates are published on the CRTC website to allow easy access by the
stakeholders and that a logging system to catalogue calls and responses be
implemented.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential to reduce stakeholder
accessibility and the CRTC ability to
monitor and assess inquiries.
Management Response:
Response Office of
Primary
Interest
Timeline
Coordinates for the Single Point of Contact for
Small Telecom Service Providers have been
posted on the Telecommunications Sector page
Director of
Public Affairs
December
2009
CRTC Operational Audit 28
of the CRTC website.
Work is currently underway to integrate the
operations of the Single Point of Contact for
Small Telecom Service Providers with the
Rapids case management system.
Director of
Public Affairs
April 2010
External service standards
The audit team reviewed the CRTC website, the Departmental Performance Report, the
Public Notice Circulars, the Treasury Board Policy on Service Standards for External
Fees and conducted interviews with managers responsible for service standards.
Treasury Board’s Policy on Service Standards for External Fees requires that
organizations involved in charging external fees must involve the stakeholders in
establishing service standards and must report annually to TBS on the performance. The
audit team noted that the stakeholders were involved in establishing the service standards
and the CRTC reports on the service standards for processing both broadcasting and
telecommunications applications through its Departmental Performance Report. The
CRTC uses Broadcasting and Telecommunication Circulars to inform the stakeholders of
the service standards and report on their achievements on the CRTC website.
Internal service standards
The CRTC has developed service standards for responding to consumer questions and
complaints. The standards are on the CRTC website. The CRTC manually monitors
these service standards to ensure compliance, but does not publish service standards
reports due to the limitation of the Rapids system to perform an age analysis. In order to
produce aging reports manual intervention is required for which resources are not
available. While there is no TB requirement for reporting on these service standards, TB
suggests that service standards performance monitoring is a best practice for government
managers. The absence of reporting on service standards reduces stakeholders’ visibility
of the effectiveness of the CRTC to address questions and complaints.
Action taken to solicit and address stakeholder comments
The audit team concludes that theCRTC is actively soliciting and addressing
stakeholders’ comments. Stakeholders’ comments have been addressed through the
Future Direction initiative and in various Public Notices.
The Operations Committee decided at its meeting of J uly 15, 2009 to defer the industry
consultations to September 2010 where industry and CRTC representatives meet face-to-
face to discuss their priorities. This decision was made because the CRTC did not want
to burden the industry with more consultations. A number of consultations and
communications with industry had occurred over the past few years: consultations had
been undertaken in fiscal year 2007/08; the 2007 Outcomes Report; the May 2008
CRTC Operational Audit 29
External Survey results; and, the TV, Radio, Accessibility Hearings undertaken in the
past two years. Also contributing to this decision was the current economic situation, the
scheduled conventional TV hearing for September 2009 and the fact that the industry was
questioning the necessity for further consultations at this time. Compensating for the
deferral of the annual consultations is the practice of the CRTC staff to have regular
meetings with industry representatives throughout the year, as well as CRTC speeches to
industry asking questions with respect to its priorities. Although not as formal, the needs
and future direction of the industry have been communicated to all those involved in the
planning process.
The audit team reviewed the consultation reports in order to map the stakeholders’
suggestions/recommendations to appropriate action plans to determine how they were
dealt with. The audit team conducted interviews with the responsible managers to
confirm that the stakeholders’ inputs are being addressed. It also met with the heads of
the Future Direction Task Forces to determine if stakeholders’ comments and suggestions
are being reflected in their action plans.
Stakeholders’ comments are captured and communicated to relevant parties through
stakeholder consultations and reports that are internally or externally initiated such as the
Telecommunications Policy Review Panel 2006 report (externally initiated), the2007
Outcomes Report (internally initiated) and the External Communication Study – June
2008 (internally initiated). The consultations are substantial undertakings. For example,
the 2007 Outcomes Report was the culmination of three roundtable consultations with
industry fee payers in Montreal, Toronto and Ottawa to discuss a proposed increase to the
CRTC operating budget for a period of five years. The J une 2008 External
Communication Study was a three-pronged effort including telephone interviews with
1,303 Canadian adults; group discussions in Vancouver, Calgary, Toronto, Montreal,
Quebec City and Halifax; and interviews with 29 senior representatives of CRTC
stakeholder groups.
Stakeholders’ comments and input are also captured on an on-going basis through central
points of contact. In addition, the Media Division scans daily news articles to keep
CRTC staff informed of news events impacting the CRTC. Questions and complaints
received from the public via telephone calls and correspondence (i.e. e-mails and letters)
are recorded in Rapids and tracked to ensure they are addressed.
Although we have not seen any evidence of the individual suggestions/recommendations
being tracked and implemented, we have noted that they are being addressed by the
appropriate Future Direction Task Forces for action. For example, suggestions on
improving the website have been addressed by the Outreach Task Force.
The suggestions and recommendations contained in the above noted reports are not
individually managed because of the work involved in tracking each recommendation.
The audit team noted that the main subjects raised by stakeholders can be characterized
as: policy review and deregulation; more timely decisions; and an improved website so
that stakeholders could find information easier.
CRTC Operational Audit 30
The audit team also noted that actions taken on stakeholders’ recommendations are found
within a number of communication vehicles (i.e. decisions, notices, bulletins, speeches
and initiatives). This fragmentation reduces the overall visibility of CRTC
accomplishments. Considering that the vast majority of regulatory stakeholders rely on
the CRTC website to obtain information, CRTC could regroup achievements and planned
activities under one easily accessible area on the website. Although the website provides
a link to speeches and various reports containing messages by the Chairman and
accomplishments, it does not provide a calendar of future public appearances of the
Chairman, Commissioners and senior managers nor a link to key messages from
management to stakeholders. Improving the transparency of initiatives and achievements
on the CRTC website would help stakeholders to better understand CRTC needs and
challenges.
Recommendation:
The Director of Public Affairs should develop a website page and/or link(s) that
report on achievements, the status of stakeholders’ recommendations provided in
the surveys and reports, and a link to a calendar of future public appearances by
the Chairman, Commissioners and members of the senior management team, as
well as a link to key messages from management contained in various reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that the public is not fully aware
of the CRTC achievements.
Management Response:
Response Office of
Primary
Interest
Timeline
The CRTC website is our number one
communication tool. The diverse nature of
visitors to the CRTC website makes it quite
difficult to have a single webpage that would
contain all of the suggested information and be
relevant to all users. With the exception of a
calendar of future appearances, most of the
identified information can indeed be found in
the “Media” section of the CRTC website. All
speeches and news releases are posted to the
website and contain our key messages. CRTC
achievements are also communicated through
the Report on Plans and Priorities and the
Departmental Performance Report. Important
regulatory information, such as decisions and
Director of
Public Affairs
Ongoing
CRTC Operational Audit 31
information bulletins, has been organized by
industry and all documents can be accessed
easily and efficiently from a number of
redundant links. Finally, a dedicated
“Consumer” section exists for key information
of particular interest to the general public.
A calendar of future appearances has not been
developed as the sensitive nature of the
Commission’s quasi-judicial role in the
industry can often necessitate last minute
changes in attendance due to a variety of
potential conflicts. The Director of Public
Affairs will however examine this proposal in
the new fiscal year with a view to providing as
much information to the public as possible.
Director of
Public Affairs
FY 2010/11
Setting & adjusting annual plans through stakeholder consultation
The audit team concluded that the CRTC has an effective stakeholder consultation
process to set and adjust annual plans.
The audit team reviewed the 3-Year Work Plan and the annual Report on Plans and
Priorities (RPP) and interviewed key personnel involved in the development of plans and
public consultations.
The CRTC undertakes a variety of consultations with stakeholders to set and adjust
annual plans. The results of the consultations are reflected in a 3-Year Work Plan and the
annual Report on Plans and Priorities (RPP). Specifically, the CRTC held industry
consultations in J anuary 2008, where industry and CRTC representatives met for a three-
day session to discuss their priorities. Other sources of information utilized by the CRTC
to determine priorities included the Outcomes Report – 2007, External Survey results –
May 2008, and TV, Radio and Accessibility Hearings undertaken in the past two years.
In addition, CRTC staff has regular meetings with Industry representatives throughout the
year to discuss priorities and annual plans, and to seek input and the Chairman delivers
speeches to the industry on licence renewals, Canadian programming, and national digital
strategy.
Interface between the Sectors
Conclusion
The audit team concludes that there is ongoing and transparent communication between
the Operations Committee (i.e. oversight body), management, Commissioners and the
CRTC Operational Audit 32
Chairman resulting in an effective interface between the sectors in support of enhanced
regulatory processes and results. This was demonstrated by the existence of an effective
internal governance structure that incorporates a number of committees that require the
involvement of the Commissioners and staff to address regulatory and management
issues. During audit interviews managers and staff indicated that there was appropriate
interface between sectors, within sectors, with Commissioners and with the functional
groups (i.e. legal, finance, human resources and communications).
Governance
The governance structure in place ensures that management’s direction, plans and actions
are appropriate and responsible. Several committees have been established to deal with
management and regulatory responsibilities. A key committee which looks at operational
and management issues is the Operations Committee. This committee meets on a weekly
basis and is comprised of the Chairman, Vice Chairman Telecommunications, Vice
Chairman Broadcasting, Secretary General, Executive Director Broadcasting, Executive
Director Telecommunications, Executive Director Policy Development and Research,
Senior General Counsel and the Director General Strategic Communications and
Parliamentary Affairs. In addition to the Operations Committee, weekly bilateral
meetings are held between the Chairman and the senior manager of each sector to discuss
operational and policy issues specific to that sector (i.e. Secretary General, Legal,
Strategic Communications, Broadcasting, Telecommunications and PDR). The
Operations Committee and the sector bilateral meetings produce summary and follow-up
documents that record decisions made, instructions provided and follow up required.
These records of decisions demonstrate management’s interventions into reallocation
decisions to support the regulatory processes and results with the resources that the
Commission has been provided.
Several other committees have been established to deal with management responsibilities
that support the Operations Committee. These include the IM/IT Steering Committee,
the HR Committee, the Steering Committees on Future Direction (i.e. streamlined rules
and regulations, effective outreach and succession/continuity planning), the Diversity and
Employment Equity Committee and the Commission’s Organizational Health and Safety
Committee. Each committee draws its membership from across the organization to
ensure all sectors are represented.
The organization’s regulatory responsibilities are discharged by a number of committees
involving the Commissioners and staff to address broadcasting and telecommunications
related issues. These committees include the Commission Full-time Members Meetings
(C/FTM), the Broadcasting Committee Meeting (BCM), the Broadcasting Committee
Meeting – Walk Around (BCM-WA), the Broadcasting Committee Review (BCR), the
Telecommunication Committee Meeting (TCM), the Telecommunication Committee
Meeting – Walk Around (TCM-WA), the Telecommunication Committee Review (TCR),
and the Senior staff members +Commissioners i.e. Operations Committee with
Commissioners (SSM+).
CRTC Operational Audit 33
The C/FTM and SSM+are chaired by the Chairman, and all Commissioners and senior
staff attend and discuss key issues. The BCM, BCM-WA, TCM and TCM-WA meet
weekly, or more frequently as required, and are chaired by the Vice-Chairperson of
Broadcasting and Telecommunications, respectively. The BCR and TCR are broadcasting
and telecommunication staff meetings and occur weekly to review incoming applications.
Both management and regulatory committees have terms of reference articulating
membership, meeting times and purpose. Whenever appropriate, committee membership
is drawn from all sectors resulting in equitable representation from across the
organization.
In order to meet the operational demands of the regulatory processes a matrix
management approach is utilized enabling groups to draw resources from other sectors to
meet both pressing and unexpected demands. For instance, the Policy, Development and
Research (PDR) Sector will utilize resources from the other sectors to participate in
research and policy development initiatives.
Perhaps the most demanding initiative requiring effective interface between the sectors is
the Future Direction initiative. The four working groups established encompass all
sectors of the CRTC by drawing resources from each sector to develop work plans and
identify key priorities. In addition, a communication plan was developed to ensure
everyone in the organization has access to information about the status of each working
group’s action plans for implementation, as well as opportunities to voice their own
concerns, questions and views.
Staff in the Legal Directorate, Finance and Administrative Services, Human Resources
and Strategic Communications is involved in regulatory and management responsibilities
through their participation in the committees established to deal with these
responsibilities. As well as their involvement in the Future Direction initiative, as noted
above, members of these sectors are also effectively interfacing with each other in their
day-to-day regulatory and management responsibilities. For instance, Finance is working
with the sectors to implement the enhanced salary management system, HR is working
with individual sectors to update HR Strategies, Strategic Communications is working
with sectors to implement the enhanced Performance Measurement Framework and the
Legal Directorate is working with other sectors in the execution of its legal
responsibilities as well as its role in the streamlining of rules and regulations.
Internal communications
The audit concluded that open and effective channels exist for internal communications
and feedback. The CRTC management has undertaken a number of initiatives to identify
and address employee concerns. This is evidenced by the internal communications
throughout the Future Direction exercise, the revised plan for the Future Direction
implementation phase, the Internal Communication Survey, focus group evaluation and
complete revamping of the Zone, as well as the recent development of an action plan to
address the results of the 2008 Public Service Employee Survey (PSES).
CRTC Operational Audit 34
Senior management regularly communicates and encourages ongoing dialogue amongst
employees. The CRTC utilizes the Zone to support the dissemination of relevant
information to employees. There is a regular newsletter (The Frequency) for all
employees to view and Weekly News that contains upcoming events, human resource
and finance news, community news, weekly messages on various topics, Chairman’s
messages, etc. In addition, strategic documentation is also available on the Zone (e.g. 3-
Year Work Plan, Future Direction initiative, HR Strategic Plan, etc.). As a result,
employees have access to the mandate, direction and priorities of the CRTC. HR and
Finance work closely with Internal Communications on a variety of HR/Finance
employee and management issues. With respect to operational and management
responsibilities, each sector participates in bi-lateral meetings with the Chairman and
other sector representatives are invited to attend as observers. This was initiated under
the present Chairman and has been well received as noted in the various interviews the
audit team has had with employees.
An Internal Communications Plan for the CRTC Future Direction initiative was
developed for fall 2008/winter 2009. The purpose of the plan was to help senior
managers continue to communicate effectively with CRTC staff – in a transparent,
diligent and coherent manner – regarding the next steps in the Future Direction strategic
exercise. As recommended in the report under the Future Direction initiative section,
progress reporting should be done in a more complete and timely manner.
Management Control Framework
Conclusion
The audit team concludes that the main elements of the Management Control Framework
(MCF) are in place and effective. This was demonstrated by strong strategic planning that
is aligned with the CRTC mandate, an enhanced performance measurement framework
that will improve DPR, the key planning elements (i.e. performance appraisals, review of
year end results, setting corporate objectives, consultations, RPP and budget approval)
having been included in the CRTC planning cycle, and strong Financial and Human
Resource management.
The audit team noted that the responsibility centre managers are not assuming their
responsibility for financial monitoring, project management should be better entrenched,
progress reporting for the Future Direction initiative should be enhanced to provide more
complete and timely reporting, and the business planning and reporting process should
combine all the core planning and reporting elements into one formal integrated process.
Background - incremental funding and SDA requirements
The CRTC is 100 per cent cost recovered through Part I Broadcasting Licence Fees and
Annual Telecommunications Fees. These fees are the only available source of funding to
address its on-going and incremental budgetary requirements. The CRTC budget had
CRTC Operational Audit 35
CRTC Operational Audit 36
been stable for a number of years. In 2007 it undertook Fee Payer Consultations and
proposed an increase to its operating budget for a period of five years. The incremental
funding was to be directed at the streamlining of processes and policies, reduced
regulatory burden and increased timeliness of service to the industry and the public. As a
result of the consultations, the CRTC was provided incremental funding for two years
(i.e. fiscal years 2007/08 and 2008/09). In fiscal year 2009/10, the incremental funding
was discontinued and CRTC was at its pre-fee payer consultation funding level.
The CRTC falls into the category of a small department
2
as established by Treasury
Board. The Auditor General recently examined governance of small federal departments
and agencies in Chapter 2 of a report tabled in early February 2009. The issue of a heavy
administrative and reporting burden was raised and two recommendations were made
with respect to easing the reporting burden and facilitating administrative shared services.
TheCRTC, like many small agencies, does not have the capacity to respond to the
reporting requirements of the government’s central agencies and is awaiting the TBS
development and implementation of an action plan to reduce the reporting burden and
facilitate a framework for shared corporate services. Until this is completed, theCRTC
must continue to leverage its capacity between the demands of its core mandate while
satisfying the reporting requirements under the reporting regime of central agencies.
The termination of the CRTC incremental funding combined with its limited capacity to
respond to the reporting requirements of the government’s central agencies further
exacerbates a situation in which the CRTC finds itself, straining to deliver its mandate in
a changing and complex environment.
During the audit engagement, staff conveyed to the audit team a feeling of being
overwhelmed by both the demands of the profound changes in the industry and the
demands of the central agencies reporting requirements. In order to maintain excellence
in the delivery of their mandate, the reporting requirements demands were neither a high
priority nor of the standard that they would like to achieve.
Strategic direction and performance measurement
The organization has a clearly defined and communicated strategic direction that is
aligned with its mandate. The CRTC 3-Year Work Plan (2008-2011), the 2009/10
Report on Plans and Priorities (RPP) and Program Activity Architecture (PAA)
adequately address the organization’s strategic outcome, priorities and related action
plans. These documents identify three priorities and articulate the planned actions over a
three year period (i.e. 2008/09, 2009/10 & 2010/11).
The CRTC has been working with Treasury Board officials to develop a more
comprehensive and cohesive program description to better reflect its overall mandate. As
a result of last year’s Management, Resources and Results Structure (MRRS) review, the
CRTC has revised its current Performance Measurement Framework (PMF). The
2010/11 RPP will have one strategic outcome, three output statements, six expected
2
Fewer than 500 employees and annual expenditures of less than $300M.
results and 16 performance indicators for the three program activities (i.e. broadcasting,
telecommunications and internal services). Under the new PMF, the CRTC will review
its performance targets mid-year and third quarter as part of the financial and non
financial review. In addition, performance measures will be reviewed on an annual basis
as part of the RPP planning process.
The audit team reviewed the revised 2010/11 PMF and found that the two weaknesses
identified by Treasury Board officials had been addressed. The CRTC revised its
performance indicators to better measure the progress towards its Strategic Outcome
(SO) and it has developed and implemented performance indicators to measure the output
levels. Performance indicators have increased from 5 to 16 for fiscal year 2010/11 and
should improve the organization’s ability to report performance in future Departmental
Performance Reports (DPR).
Planning cycle
The audit team reviewed the key elements of the CRTC planning cycle (i.e. performance
appraisals, review of year end results, setting corporate objectives, consultations, RPP
and budget approval). It found that the employee performance appraisals and year end
results were evaluated and the results were incorporated into this year’s employee
objectives. Although employee objectives had not yet been formally approved by the
Chairman at the time of the audit, the audit team was satisfied that the draft objectives
were reflective of the CRTC strategic direction for the year.
As noted above, the CRTC has been working with Treasury Board officials to develop a
more comprehensive and cohesive program description to better reflect its overall
mandate. A revised PMF has been developed and is being implemented in the RPP
which should improve the organizations ability to report performance in future DPR.
The budget approval process for FY 2009/10 was very rigorous. FY 2008/09 was the last
year for CRTC incremental funding; therefore, an A-base review process was required to
establish both salary and operations & maintenance (O&M) funding for the coming fiscal
year. A number of salary scenarios including the financial impact were presented to the
Operations Committee. The basic objective was to allow the organization to carry out its
mandate and minimize the risk of job loss. In the end, the scenario that allowed the
CRTC to meet both these objectives was selected and a notional salary budget allocation
was set by the Operations Committee at its J anuary 26, 2009 meeting. For the O&M
funding, costing information based on actual expenses for the past three years by
responsibility center and sector was prepared and distributed to each sector. Each sector
was asked to identify the amount they considered necessary as a base operational and
maintenance allocation. The total of all sectors could not exceed the overall set base. A
number of budget meetings held between finance and the sectors ensued, and finally at
the Operations Committee meeting of May 19, 2009 the operating budget, including both
salary and O&M funding allocations was approved.
CRTC Operational Audit 37
The audit team concludes that activities and resources to achieve the organization’s
objectives have been integrated into the budget, a formal process is in place to review and
set resource allocations, and a timely budget is developed at the appropriate level of
detail.
Business planning and reporting process
There is no integrated business planning and reporting process that combines all the core
planning and reporting elements into one formal process. The audit team found that
many of the planning and reporting elements are available individually; however, they
are not collectively brought together through the application of an integrated business
planning and reporting process. The audit team found that the current business planning
and reporting process is effective; however, the efficiency of the process can be
enhanced.
The current planning process consists of the 3-Year Work Plan, EX Performance
Agreements, Reports on Plans and Priorities (RPP) and budget and resource allocation.
The reporting process consists of periodic reviews of financial information, Departmental
Performance Report (DPR), EX performance reviews, employee performance feedback
and employee learning plans.
To ensure that the output of one process becomes the input to the next and little or no
duplication of effort occurs within the planning community and individual sectors, the
business plan should include both the IT and communication strategies and associated
costs, the performance measurement framework targets and planned mid-year and third
quarter reviews, potential sources of funding and potential risks that could negatively
impact the CRTC’s strategic direction. By including all of the above in the business
planning and reporting process, each item would be assessed throughout the year as part
of the financial and non financial reviews by the Operations Committee. This would
ensure that the status of the planning elements are periodically reviewed and discussed at
the Operations Committee meetings.
To facilitate the periodic discussions a pre-set forward agenda could be developed to
ensure all major planning and reporting elements are reviewed at specified times
throughout the year. The implementation of a pre-set forward agenda that both identifies
and sets a time for pertinent planning and reporting elements (i.e. IT/IM Strategy,
Communication Strategy, HR Strategy, performance targets, funding and risk
management, EX Performance Agreements and performance reviews) to be reviewed by
the Operations Committee would ensure that all planning and reporting elements are
considered at the appropriate time for discussion and decision, thereby ensuring that the
output of one planning element becomes the input to the next planning element process
with little or no duplication of effort.
The audit team recognizes that the strategic plan must be very dynamic to meet the
demands of external influences which can change over night. As an example, an
unplanned public hearing process can be initiated for an issue that was not foreseen only
CRTC Operational Audit 38
months earlier. The CRTC identifies and intuitively assesses risks and mitigating
strategies for management decisions and regulatory planning with respect to its three
program activities – broadcasting, telecommunications and internal services; however,
they are not collectively documented and discussed in the business planning and
reporting process.
Presently the business planning and reporting process does not identify and document
possible risks and associated mitigating strategies such as hiring temporary staff,
reducing service levels, etc. to offset any impact on staff. TheCRTC must react to these
occurrences and the consequences of their decision may not be readily known and
understood fully without previously assessing the risk and documenting the mitigating
strategies. Mitigating strategies would consider the impact on all other activities that
could be impacted by unplanned hearings. Without the road map of a comprehensive
business planning and reporting process, senior management will not always be fully
equipped to assess the impact on already committed resources and priorities.
An integrated business planning and reporting process that combines all the planning and
reporting elements will enable senior management to fully assess financial and non
financial information and the impact on all CRTC activities, thereby maintaining a
strategic focus throughout the year.
Recommendation:
The Director General of Strategic Communications and Parliamentary Affairs,
in collaboration with the Secretary General and the Executive Directors of
Broadcasting, Telecommunications and PDR, should develop and implement an
integrated planning and reporting process and establish a pre-set forward
agenda to ensure that all major planning and reporting elements are reviewed at
specified times throughout the year.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential that senior management will not
maintain a strategic focus throughout the
year.
Management Response:
Response Office of Primary
Interest
Timeline
A CRTC Annual Corporate Planning and
Reporting cycle has been developed in
consultation with all sectors of the CRTC. It
will be presented for the approval of the
Operations Committee.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010
CRTC Operational Audit 39
Development and approval of the cycle is
the first phase in establishing an integrated
planning and reporting process throughout
the organization. Additional processes will
be identified and incorporated into the
process over the next few months. (For
example, further input from members of the
Corporate Management Committee will be
solicited in J anuary 2010.)
Director General,
Strategic
Communications
and Parliamentary
Affairs
FY 2010/11
Financial management/monitoring
Financial management policies and authorities are established and communicated.
Policies are available on the intranet and/or links are made to the TB site for financial
policies. Financial policies and authorities are known and understood.
Responsibility for financial monitoring is included in each manager’s Performance
Management Agreement. Each Executive Director has a financial management
commitment with appropriate performance measures (i.e. budget allocations are aligned
with CRTC priorities; commitments are updated on a monthly basis; and expenditures are
monitored monthly and do not exceed available budget allocations). Each sector head
and their management team are now fully responsible and accountable for all aspects of
budget management for their sector. Finance pre-approval for job requisitions is no
longer required, as it was in fiscal 2008/09. It is now the responsibility of each sector
head to ensure that all staffing actions undertaken are within their approved salary budget
and that salary forecasts are updated on a monthly basis. To assist managers to achieve
the performance measure, the Finance Division has established procedures and provides
reports to each Executive Director, and financial staff members assist each sector on a
monthly basis.
The first quarter of fiscal year 2009/10 was not completed on time by the managers since
this was a transition year where the salary budget shifted from the Finance Directorate to
Responsibility Centre Managers. Rather than present an unrealistic budget summary to
the Operations Committee, the Finance Directorate decided to defer the J uly 31, 2009
first quarter reporting to allow additional time for Responsibility Center Managers to
input the required commitments. As a result, the budget summary report presentation to
the Operations Committee occurred in August 2009, resulting in a time span of almost
five months before senior management was provided with an overview of the CRTC
2009/10 spending.
This time span is in direct contrast to fiscal year 2008/09, where the financial reporting
package was presented to the Operations Committee less than 15 calendar days after
period end and was presented to management ten or more times per year. The timeliness
issue is likely attributable to the shift of responsibility from the Finance Division to the
CRTC Operational Audit 40
Executive Directors who may be experiencing problems as they implement the new
practice with respect to salary and O&M budget allocations and management.
Recommendation:
The Director General Finance and Administrative Services should continue to
reiterate to the Responsibility Centre Managers their responsibility for financial
budgeting and monitoring to ensure that they understand and provide complete,
accurate and timely financial information for the monthly budget summary
report.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential that financial information
reported to senior management is not
complete and timely.
Management Response:
Response Office of Primary
Interest
Timeline
Procedures are in place whereby Finance
staff advises sector heads of the requirement
to review and update all their commitments
(Salary and O&M) on a monthly basis
before month end. In addition, as part of
these monthly notifications Finance staff
have advised sector heads that additional
financial training to sector management
teams would be provided upon request.
Director General,
Finance and
Administrative
Services
Ongoing
The requirement to do a comprehensive and
monthly review of all financial
commitments was recently re-emphasized
by the Secretary General (Chief Financial
Officer) to the Executive Committee as part
of the FY 2009/10 mid-year budget review.
Secretary General November
2009
Executive performance agreements have
been updated to reflect the expectation that
all members of the Executive Committee
and their management teams will review
and update their commitments monthly.
Director General,
Human Resources
Completed
Human resource management
The audit team concludes that human resource planning is aligned with strategic and
business planning, enabling theCRTC to identify current and future resource
CRTC Operational Audit 41
requirements and analyzes them against existing human resource competencies and
capacities.
The HR Strategic Plan reflects the conclusions and identifies strategies to implement
priorities identified by the CRTC Future Direction Task Force “Innovative Approaches to
People and Succession Planning” initiated in 2007/08. The HR Plan articulates the
following seven key initiatives: build capacity for key leadership positions, build
workforce capacity, maintain and develop current workforce, meet changing business
needs, build and maintain bilingual capacity for key leadership positions, build a diverse
and representative workforce, and renewal. Each initiative is broken down into activity,
responsibility, indicators, objective, status and comments and is monitored by HR on a
regular basis. Currently each sector is updating its section of the plan and the overall
results will be presented to the Operations Committee this fall.
Recruitment plans have been put in place that addresses the need to build capacity for the
key EX leadership positions. In May 2009 the EX shortage was addressed with the EX1
and EX2 competition and subsequent staffing actions.
The audit team concludes that the organization has an effective performance management
program for all of its employees. Responsibilities and performance expectations to which
staff are accountable are formally defined and clearly communicated. J ob descriptions
and performance agreements exist for this purpose and are up-to-date. Audit testing
indicated that performance management agreements (i.e. EX Group) and performance
feedback agreements (i.e. Non EX Group) are aligned with the 3-Year Plan and the RPP.
Performance management agreements contain clear and assessable commitments,
including HR and financial management while performance feedback agreements have
attainable work objectives. Performance training plans are established and implemented
with clear accountability and all staff has access to training and support in the
performance evaluation and feedback process. An effective review mechanism is in
place to ensure equity and consistency in determining ratings.
Future Direction initiative
The CRTC executive team, at its November 2007 retreat, discussed its future direction
and how it could rise to the challenges over the next five years – covering its purpose,
core business and key focus areas. The challenges were the result of the profound change
to the communication industry as new digital technologies transform how Canadians
communicate, how theCRTC delivers its mandate and how the public is informed and
entertained.
Four internal task forces were created to consult, conduct further analysis and develop
action plans for the four key areas, each under the leadership of a senior CRTC executive.
These areas are:
• Structures that address convergence;
• Focused and streamlined rules and regulations;
CRTC Operational Audit 42
• Achieving effective outreach; and,
• Innovative approaches to people and succession issues.
Each task force prepared an action plan that was presented to the Operations Committee
in J une 2008 and presented to all CRTC employees at an all-staff meeting on J uly 12,
2008, after which each task force commenced working on its assigned area of focus.
Future Direction was a major internal initiative that further increased demands on staff
time. The additional workload was completed without staff augmentation and in addition
to the demands of their day jobs.
During the audit team’s initial interviews with managers, they expressed a concern that
the Future Direction initiative had lost momentum. During the examination phase of the
audit, the team assessed whether this was caused by inadequate communication, team
leads not providing the status on a regular basis or overall project management
responsibilities not clearly articulated and assigned.
a) Communication
A review of the Zone indicated that there had been a number of communications on the
Future Direction (FD) initiative. They were as follows:
• April 9, 2009 - FD Status Report on Goals & Milestones;
• Winter 2009 – Quarterly Reports from the Champions;
• Fall 2008 – Quarterly Reports from the Champions;
• October 2008 – Succession planning initiative;
• September 2008 – Continuity Plan; and,
• J uly 2008 – Conclusions from retreat presented to employees since the initiative
began in May 2008.
Since the Town Hall meeting in J uly 2008, the Zone indicated that six communications
on the initiative had been published; however, the audit team noted that the “Winter 2009
- Quarterly Report from the Champions” had not been published. The audit team was
told that the March 12, 2009 update to the Operations Committee included an update of
only one key area, “Innovated approaches to people and succession planning”. The audit
team was informed that it was not published on the Zone because of an oversight. The
audit team reviewed the update briefing material provided to the Operations Committee
and concluded that this was correct because it was a good news story. The Operations
Committee was not provided updates for the other two key areas still active (i.e. Outreach
and Streamlining).
The audit team concluded that the communication of the Future Direction initiative could
be enhanced to ensure it is complete and timely.
CRTC Operational Audit 43
Recommendation:
The Champions, in collaboration with their project leads and the Director
General of Strategic Communication and Parliamentary Affairs, should ensure
that the Quarterly Reports on Future Direction are consistently published in a
complete and timely manner.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that incomplete reporting may
cause a decline in employee support for
the initiative.
Management Response:
Response Office of Primary
Interest
Timeframe
The Director General of Strategic
Communications and Parliamentary Affairs
will reiterate the importance of quarterly
reporting to the project leads and ensure that
all information is provided in a complete and
timely manner.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010 and
ongoing
Publication dates for quarterly reports will
be pre-set and the Strategic Communications
sector will ensure that all required
information is submitted in conformity with
the relevant timelines.
Director General,
Strategic
Communications
and Parliamentary
Affairs
April 2010
and ongoing
b) Project management
The audit team reviewed the action plans developed for each of the four key areas and
interviewed responsible managers to assess the progress since the action plans were
presented at the all-staff meeting on J uly 12, 2008.
i) Key Area - Structures that address convergence
The key area had been implemented shortly after the all-staff meeting. It was a critical
element that needed to be completed first to provide a solid organization structure to
move ahead with the other three key areas. Activities common to both Broadcasting and
Telecommunications became the responsibility of the new sector Policy Development
and Research (PDR) and organization charts were restructured.
The audit team concluded that this key area has been successfully implemented as
evidenced by interviews with responsible managers and staff, and review of pertinent
documentation.
CRTC Operational Audit 44
ii) Key Area - Innovative approaches to people and succession planning
The key area is comprised of the following five activities: succession/continuity
planning; job standardization; job rotation; leadership training; and the “CRTC
University” (CRTC-U).
The majority of tasks under the succession/continuity planning activity are completed.
Key leadership positions have been identified and staffing for EX-01 and EX-02
positions has been carried out. The talent management plans for EX, EX equivalent and
EX minus one groups have been developed and were reviewed as part of the annual
performance review. The talent management exercise will be extended to all other
employees, and managers will be responsible for establishing development plans with
their employees as well as ensuring continuity within their own operations. At the time
of the audit it was expected to be completed in fall 2009.
A consultant had been selected to create a customized leadership development program
for CRTC managers that will focus on people skills and people management. It is
expected to be ready for delivery in fall 2009.
The CRTC-U training program includes an orientation program for new employees as
well as a development program for existing employees. A sub-committee of CRTC
employees was struck and it is expected to roll-out CRTC-U in fall 2009.
Although not all activities have been implemented, the audit team concludes that this key
area is making effective progress and is adequately managed as evidenced by interviews
with responsible managers and staff, and review of pertinent documentation.
iii) Key Area - Achieving Effective Outreach
The key area is comprised of three activities: engage and educate smaller groups in the
industry; create a better understanding of the CRTC mandate and activities for members
of the government and the public; and, create an awareness of the CRTC with the 18 to
34 year age groups. Portions of the activities are either completed or works-in-process.
Budget concerns are impacting some of the activities.
Although not all activities have been implemented, the audit team concludes that the key
area is making effective progress and is adequately managed as evidenced by interviews
with responsible managers and staff, and review of pertinent documentation.
iv) Key Area - Streamlined Rules and Regulations
Prior to the Future Direction initiative a government policy direction was issued in
December 2006 to foster increased reliance on market forces in the telecommunications
CRTC Operational Audit 45
industry. The directive led to the initiation of a number of activities within the CRTC
Telecommunications sector. Separately under the Future Direction initiative, additional
activities were initiated relating to a number of process improvements and changes in the
regulatory framework, including the removal of some existing rules.
These two initiatives triggered a number of activities within the CRTC to address the
common overall objective of a more simplified regulatory process. One of the initial
actions undertaken by the key area project manager was to identify and document all the
activities carried out by the Commission which necessarily captured those generated by
the policy direction. The purpose of the activity catalogue was to identify further process
improvement opportunities. The activities were documented in the activity catalogue and
divided by sector – Telecommunications, Broadcasting, PDR and Corporate Services and
Operations. While the activities initiated within the telecommunications sector that
addressed the implementation of the government policy direction formed part of the
activities catalogue, they continued to be implemented by the telecommunications
directorate, independently of the Future Directions initiative.
The audit team reviewed the activity catalogue and completed a comparison of the
activities recorded in the catalogue to the April 9, 2009 Future Direction Status Report on
Goals & Milestones (i.e. Quarterly Report) published on the Zone. As the April Status
Report dealt only with the activities that were retained as priority items to be addressed at
the Future Direction Workshop held in May 2008 and the Public Hearing Process Charts
prepared by a consulting firm, not all activities initially recorded in the activity catalogue
were included in the Quarterly Report and not all activities underway outside of the
Streamlining initiative were included in the catalogue.
The results of the comparison were discussed with the responsible managers and it was
agreed that activities underway for which no status was provided included 19
telecommunications activities, 8 PDR activities and 15 broadcasting activities. The audit
team interviewed those responsible for the activities not reported and not included in the
catalogue. They stated that these activities were at various stages of completion;
however, they had not been directed to provide periodic updates.
The project manager for the Streamlining initiative reported that when he created the
activities catalogue it was never envisioned that the catalogue would become a living
document which would be updated overtime. Rather it was a tool developed for the
initial stages of the Streamlining project to identify the processes most in need of
improvement.
As a result, only some of the activities being carried out in various parts of the
Commission were found in both the Quarterly Report and the Activity catalogue.
However, the audit team was satisfied that staff responsible for the individual activities
was managing them adequately.
The audit team concludes that the following factors have contributed to employee
concerns that the initiative had lost momentum:
CRTC Operational Audit 46
• The CRTC did not develop a formal monitoring and reporting framework for
activities under the streamlining initiative;
• The activity catalogue was not developed for the purpose of providing centralized
monitoring of process improvements;
• There may be lost opportunities for possible cross-sector integration of common
activities; and,
• The performance objectives of the project manager responsible for the key area
only referred to the combined rules of procedures, which is only one activity of
the key area.
These factors resulted in the lack of complete and timely weekly reporting as set out in
the terms of reference as well as quarterly reporting as set out in the Future Direction
initiative communication plan.
Recommendation:
The Future Direction Champion and Project Manager for Streamlined Rules
and Regulations, in collaboration with the Executive Directors should:
• Identify the activities that constitute the Future Direction
streamlining and policy direction initiatives underway;
• Identify possible activities that can be integrated;
• Update the activity catalogue on a regular basis; and,
• Clearly establish responsibility for the management of the key
area as well as other initiative and ensure that complete and
timely bi-weekly progress reporting is implemented.
Risk Type Audit Risk Rating Impact
Operations
Risk
Minor Potential that not all activities are
effectively managed.
Management Response:
Response Office of
Primary
Interest
Timeframe
Priorities for the Streamlined Rules and
Regulations initiative were initially
established through a Process Mapping
exercise that identified process “hot spots”
most in need of attention. Remaining
activities will continue to be reviewed and re-
prioritized on an ongoing basis according to
operational requirements and resources.
Future Direction
Champion,
Streamlined
Rules and
Regulations
Ongoing
CRTC Operational Audit 47
Project management will collaborate with
Strategic Communications to ensure that
quarterly status updates are issued on a
regular and timely basis.
Project
Manager,
Streamlined
Rules and
Regulations
J anuary 2010
and ongoing
Project scope and management responsibility
will be further clarified through the upcoming
development of Future Direction Exercise -
Phase II.
Future Direction
Champion,
Streamlined
Rules and
Regulations
April 2010
CRTC Operational Audit 48
RISK ASSESSMENT FRAMEWORK ANNEX 1
Risk Types
Audit risk types have been classified according the COSO Internal Control-Integrated
Framework as follows:
Strategy: High-level goals, aligned with and supporting the CRTC mandate
Operations: Effective and efficient use of resources
Monitoring: Accurate assessments or evaluation of activities
Reporting: Reliability of operational and financial reporting
Compliance: Compliance with applicable laws, regulations, policies and procedures
Risk Ratings
Audit findings are rated as follows:
Major: A key control does not exist, is poorly designed or is not operating as intended
and the risk is potentially significant. The objective to which the control relates is
unlikely to be achieved. Corrective action is needed to ensure controls are cost effective
and/or objectives are achieved.
Moderate: A key control does not exist, is poorly designed or is not operating as
intended and the related risk is more than inconsequential. However, a compensating
control exists. Corrective action is needed to avoid sole reliance on compensating
controls and/or ensure controls are cost effective.
Minor: A weakness in the design and/or operation of a non-key process control. Ability
to achieve process objectives is unlikely to be impacted. Corrective action is suggested
to ensure controls are cost effective.
CRTC Operational Audit 49
LIST OF POSITIONS INTERVIEWED ANNEX 2
Position Title
Vice-Chairman, Broadcasting
Vice-Chairman, Telecommunications
Executive Director, Broadcasting
Executive Director, Telecommunications
Executive Director, Policy Development & Research
Senior General Counsel
Secretary General
Senior Advisor to the Secretary General
Director General, Television Policy & Applications
Director General, Policy, Decisions & Operations
Director General, Competition, Costing & Tariffs
Director General , Strategic Communications & Parliamentary Affairs
Director General, Human Resources
Director General, Finance & Administration Services
Director, Convergence Policy
Director, Industry Analysis
Legal Counsel (s)
Director, Planning & Process
Director, IM/IT
Director, Public Affairs
Senior Communications Officer Stakeholder Relations
Director, Dispute Resolution
Director, Communication Policy (Telecom)
Director, Ownership and Acquisitions (PDR)
Director, Decisions and Operations (Telecom)
Senior Director, Alternate Dispute Resolutions
Senior Director, Distribution Policy & Applications
Director, Broadcasting, Streamlining and Decisions
Correspondence & Systems Coordinator
A/Manager, Data Collection and System
Manager, Web Publishing Policy & Standards
Assistant Director IT, Business Solutions Center
Client Relationship Manager
Security Architect
IT Security Coordinator
Programmer/Analyst
Manager, Information Resource Centre
Information Management Policy & Planning Analyst
Manager, Ownership Data & Management
Radio Monitoring Coordinator
CRTC Operational Audit 50
Specialist and Systems Analyst
Director, Dispute Resolution
Director, Communication Policy (Telecom)
Director, Ownership and Acquisitions (PDR)
Director, Decisions and Operations (Telecom)
Director of Media & Parliamentary Relations
Telecom Analyst /Telecom Contact
Manager, Broadcasting Process and External Liaison
Manager, Parliamentary Relations, Corporate Planning and Reporting
Senior Communications Officer, Parliamentary Relations
Chief, Human Resources Operations
Chief, Human Resources Programs & Planning
Human Resource Program Officer
Assistant Director, Financial Policy, Planning & Systems
Senior Financial Planning & Reporting Officer
Special Advisor, Operations
Chief Coordinator, Chairman’s Office
Senior Analyst, Competitive Disputes
CRTC Operational Audit 51
AUDIT OBJECTIVES AND AUDIT CRITERIA ANNEX 3
Audit Objective 1
The effectiveness and efficiency of regulatory processes and practices (e.g. public
hearing processes, application processes, scheduling, etc.), and to ensure that
services are delivered in a fair, open, transparent and timely basis and in a manner
that minimizes the regulatory burden on all stakeholders. This will include an
assessment of the on-going and planned CRTC initiatives to enhance the
regulatory processes and service delivery.
Audit Criteria
1. The organization has resources to support research and policy analysis.
2. The organization has access to adequate subject-matter expertise for the conduct
of policy research.
3. Responsibility for research, consultation and analysis of policy options and
related impacts on program is clearly and formally delineated.
4. A process exists to solicit stakeholder and employee comments related to
improvements in service delivery.
5. Significant change initiatives and management actions are communicated to the
appropriate stakeholders on a timely basis.
6. When the CRTC wishes to develop or change a policy, the CRTC adopts a
public notice describing the proposed changes. A deadline is specified for
comments and reply comments.
7. CRTC proceedings attract participation from opposing interests, so it normally
informs critiques of industry positions.
8. Public interest groups participate in appropriate proceedings.
9. All comments and reply comments are made a part of the public record of the
proceeding.
10. Only in very limited circumstances does the CRTC permit parties to attend its
proceedings to submit confidential material.
11. The CRTC makes decisions public and explains the rationale for their
decisions.
12. Members of the public that wish the CRTC to change its rules or develop new
ones may file appropriate Forms requesting that action. The CRTC puts such
petitions out for comment and reply comment before deciding whether to issue
ruling.
13. The CRTC receives, processes, and assists in the resolution of consumer
complaints. This process is well documented.
Audit Objective 2
The effectiveness of existing information systems that are used to collect and
provide information in support of the CRTC regulatory processes (e.g. the
reliability of information systems for decision-making and accountability
CRTC Operational Audit 52
purposes).
Audit Criteria
1. Records, data and information are appropriately secured in compliance with
privacy legislation.
2. The organization leverages information technology to enhance user service and
access.
3. Transactions are coded and recorded accurately and in a timely manner to
support accurate and timely information processing.
4. Controls are in place to ensure accuracy of transaction coding and processing
(e.g., batch totals, reconciliations, supervisory review, management approval, etc.).
5. Records and information are maintained in accordance with laws and
regulations.
6. Assets and records are periodically verified.
7. Processes and procedures exist to support the continuity of information and
systems.
8. Development, implementation or changes to information systems are based on a
strategic plan for information systems and responsive to achieving organizational
strategic and operational objectives.
9. Implement IT solutions that align with the CRTC business environment, policy
goals, and statutory requirements.
Audit Objective 3
The effectiveness of monitoring activities (including performance monitoring
reports) and the extent to which the results of monitoring activities are used for
regulatory policy and management decision making.
Audit Criteria
1. External and internal environments are monitored to obtain information that may
signal a need to re-evaluate the organization’s objectives, policies and/or control
environment.
2. Monitoring of policy and program design options occurs in a regular and timely
manner.
3. Evaluation activities are used to identify policy and program strengths,
weaknesses and impacts (intended and unintended), as well as alternative ways of
designing policies, programs and initiatives.
4. Financial and non-financial reporting is reviewed and approved.
5. Appropriate and timely financial and non-financial reporting is communicated
internally and externally.
6. Management has identified appropriate performance measures linked to planned
results.
7. Management monitors actual performance against planned results and adjusts
course as needed.
CRTC Operational Audit 53
Audit Objective 4
An assessment of the effectiveness of on-going and planned initiatives that address:
a) the adequacy of information provided to regulatory stakeholders on the CRTC
website (i.e. to inform them of CRTC plans, processes, service delivery standards,
documentation requirements and results achievement): b) the CRTC actions taken
to address stakeholders comments received as part of industry surveys (e.g. dealing
with issues such as stakeholder satisfaction and recommendations for process
improvement): and c) the current processes of consultation with the CRTC
stakeholders in the setting and adjusting of the CRTC annual plans (e.g. is the
CRTC asking its stakeholders (industry, public) the right questions and is it
capturing the right information?).
Audit Criteria
1. Lines of communication exist between the organization, and stakeholders.
2. Suggestions, complaints and other input are captured and communicated to
relevant internal parties.
3. Input is sought, on a regular basis, from users and other stakeholders through
mechanisms such as environmental scanning, monitoring of public opinion and
client satisfaction.
4. Reaction to feedback from external stakeholders is managed in a coordinated
manner throughout the organization.
Audit Objective 5
An assessment of the effectiveness of interfaces by the four sectors, with other
sectors of the CRTC (e.g. legal, strategic communications and commission
members) in support of enhanced regulatory processes and results.
Audit Criteria
1. There is ongoing and transparent communication between the Operations
Committee (i.e. oversight body), management, Commissioners and the
Chairperson.
2. An information-sharing process exists to support the efficient and targeted
dissemination of relevant and reliable information to those that need it.
3. Open and effective channels exist for internal communications and feedback.
4. A clear and effective organizational structure is established and documented.
5. Memoranda of understanding, terms of reference or equivalent documents exist
for those government-wide or horizontal initiatives to which the organization
contributes.
Audit Objective 6
The effectiveness of CRTC management framework (e.g. practices and procedures
CRTC Operational Audit 54
relating to planning, organizing, controlling, leading, communications and
management of human and financial resources) within each of the sectors noted
above.
Audit Criteria
1. The CRTC has clearly defined and communicated strategic directions and
strategic objectives, aligned with its mandate.
2. The CRTC has in place operational plans and objectives aimed at achieving its
strategic objectives.
3. Human resource planning is aligned with strategic and business planning.
4. An HR Plan is documented and communicated and includes the following
elements:
• analysis of current and future resource and competency needs;
• analysis of key positions and succession planning; and
• training and development plan.
5. The CRTC identifies its current and future resource requirements and analyzes
them against existing human resource competencies and capacities.
6. The CRTC provides employees with the necessary training, tools, resources and
information to support the discharge of their responsibilities.
7. The CRTC has in place a system for the performance evaluation of employees.
8. Open and effective channels exist for internal communications and feedback.
9. The activities, schedules and resources needed to achieve objectives have been
integrated into the budget.
10. Financial management policies and authorities are established and
communicated.
11. Compliance with financial management laws, policies and authorities is
monitored regularly.
12. Management compares results achieved against expectations, on a periodic
basis.
13. Responsibilities and performance expectations to which managers and
supervisors are held accountable are formally defined and clearly communicated.
J ob descriptions and/or performance agreements should exist for this purpose and
be up-to-date.
CRTC Operational Audit 55
SUMMARY: RECOMMENDATIONS AND ANNEX 4
MANAGEMENTS RESPONSES
Recommendation #1
The Director of IM/IT should conduct a risk assessment on all CRTC key
databases and the databases considered to be at risk should be subject to a
Privacy Impact Assessment.
Risk Type Audit Risk Rating Impact
Compliance Moderate Potential that personal information could
be provided in a way that is not compliant
with the Privacy Act. Personal
information could be disclosed.
Response Office of
Primary
Interest
Timeline
A consultant will be hired to conduct a Threat and
Risk Assessment (TRA) on the DCS system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a Privacy
Impact Assessment (PIA) on the OWN system.
Director
IM/IT
October
2009
A consultant will be hired to conduct a TRA on
the CRTC network.
Director
IM/IT
Summer
2010
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be carried
out on those priority systems revealed to be at
significant risk.
Director
IM/IT
Summer
2010
All remaining databases will undergo PTRAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
The IM/IT Steering Committee will identify
priority systems to undergo Preliminary PIAs
(PPIA). Full PIAs will subsequently be carried out
on those priority systems revealed to be at
significant risk.
Director
IM/IT
October
2010
All remaining databases will undergo PPIAs as
they are modified according to the existing project
life cycle. Business cases presented to the IM/IT
Steering Committee during the project proposal
cycle will include explicit reference to the
importance of carrying out these assessments.
Director
IM/IT
December
2010 and
ongoing
CRTC Operational Audit 56
Recommendation #2
The Director of IM/IT, in collaboration with senior management, should:
• Conduct a risk assessment to determine whether or not the CRTC is at
risk by not having an alternate data center processing site. If the risk
is deemed unacceptable, the CRTC should identify an appropriate
location to set up an alternate data processing site; and,
• Test the Business Continuity Plan and document the results.
Risk Type Audit Risk Rating Impact
Operations Minor Potential that CRTC could not provide
service to its clients and the public.
Response Office of
Primary
Interest
Timeline
IM/IT staff will test the current Business
Continuity Plan (BCP) to determine whether it
meets legislative and operational requirements.
Results and associated recommendations will be
presented to the Operations Committee.
Director,
IM/IT
Fall 2010
Recommendation #3
The Director of IM/IT should implement a rigorous monitoring and reporting
regime on the performance of the data centre.
Risk Type Audit Risk Rating Impact
Monitoring Moderate Potential process problems are not
identified for remedial action at an early
stage.
Response Office of
Primary
Interest
Timeline
A monitoring system called System Center
Operations Manager (SCOM) has been
implemented to record service information
related to the data center.
Director, IM/IT Spring 2009
Best practices for monitoring and reporting
regimes are currently being reviewed.
Director, IM/IT Spring 2010
Appropriate monthly monitoring and reporting
system to be implemented as required.
Director, IM/IT Fall 2010
CRTC Operational Audit 57
Recommendation #4
The Director of IM/IT should update the CRTC list of risks by conducting a Threat
and Risk Assessment of all services.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential security threats go undetected.
Response Office of Primary
Interest
Timeline
Director IM/IT The IM/IT Steering Committee will identify
priority systems to undergo Preliminary TRAs
(PTRA). Full TRAs will subsequently be carried
out on those priority systems revealed to be at
significant risk.
Summer
2010
Director IM/IT All remaining databases will undergo PTRAs as
they are modified according to the existing
project life cycle. Business cases presented to
the IM/IT Steering Committee during the project
proposal cycle will include explicit reference to
the importance of carrying out these
assessments.
December
2010 and
ongoing
Recommendation #5
The Director of IM/IT should enhance the IM/IT Roadmap by identifying specific
projects, their estimated costs and how each project ties into the IT strategic
objectives, and the 3-Year Work plan.
Risk Type Audit Risk Rating Impact
Strategy Minor Potential that resources are used on
projects that do not further the objectives
of the CRTC.
Response Office of
Primary
Interest
Timeline
All suggested information will be incorporated
into the IM/IT Roadmap during the next project
proposal cycle, with projects for the next 3 years
clearly linked to the CRTC priorities and 3-Year
Work Plan through the use of business case
templates.
Director, IM/IT Summer
2010
CRTC Operational Audit 58
Recommendation #6
The Director of IM/IT should include in all project business cases the following
elements:
• Summary of alternatives considered;
• Resources required;
• Identification of risks for not approving the project;
• Identification of risks with mitigation strategies for the project;
• Reason(s) for selecting the recommended solution; and,
• Project management accountability framework.
Risk Type Audit Risk Rating Impact
Operations Moderate Potential that not all pertinent information
is assessed before making the investment
decision.
Response Office of
Primary
Interest
Timeline
All future business cases presented to the IM/IT
Steering Committee will adhere to the existing
Business Case standards identified by the
auditors.
Director, IM/IT Summer
2010
Recommendation #7
The Executive Directors of Broadcasting and Telecommunications, in
collaboration with the Executive Director of the Policy Development and
Research, should ensure that the quarterly reports on service standards are
produced on a timely basis. Should reports be delayed, then the CRTC website
should provide a notice concerning the delayed reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential negative publicity regarding the
CRTC business processes.
Response Office of
Primary
Interest
Timeline
The CRTC has an obligation to report on an
annual basis, as set out in the Treasury Board
Policy on Service Standards for External Fees.
CRTC policies referring to quarterly reporting
have since been superseded by more recent
developments. While increased reporting
frequency is desirable, resource limitations have
led to the elimination of quarterly publication of
Executive
Directors
Ongoing
CRTC Operational Audit 59
service standard reports. The latest annual
reports include quarterly information and can be
found on the CRTC website. Regardless,
contracts for the automation of the reporting
process have been issued. Such automation may
permit a return to a quarterly reporting
frequency in the future.
Recommendation #8
The Director of Public Affairs should ensure that the telecommunication
coordinates are published on the CRTC website to allow easy access by the
stakeholders and that a logging system to catalogue calls and responses be
implemented.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential to reduce stakeholder
accessibility and the CRTC ability to
monitor and assess inquiries.
Response Office of
Primary
Interest
Timeline
Coordinates for the Single Point of Contact for
Small Telecom Service Providers have been
posted on the Telecommunications Sector page of
the CRTC website.
Director of
Public Affairs
December
2009
Work is currently underway to integrate the
operations of the Single Point of Contact for
Small Telecom Service Providers with the Rapids
case management system.
Director of
Public Affairs
April 2010
Recommendation #9
The Director of Public Affairs should develop a website page and or link(s) that
report on achievements, the status of stakeholders’ recommendations provided in
the surveys and reports, and a link to a calendar of future public appearances by
the Chairman, Commissioners and members of the senior management team, as
well as a link to key messages from management contained in various reports.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that the public is not fully aware
of the CRTC achievements.
Response Office of Timeline
CRTC Operational Audit 60
Primary
Interest
The CRTC website is our number one
communication tool. The diverse nature of
visitors to the CRTC website makes it quite
difficult to have a single webpage that would
contain all of the suggested information and be
relevant to all users. With the exception of a
calendar of future appearances, most of the
identified information can indeed be found in the
“Media” section of the CRTC website. All
speeches and news releases are posted to the
website and contain our key messages. CRTC
achievements are also communicated through the
Report on Plans and Priorities and the
Departmental Performance Report. Important
regulatory information, such as decisions and
information bulletins, has been organized by
industry and all documents can be accessed
easily and efficiently from a number of
redundant links. Finally, a dedicated “Consumer”
section exists for key information of particular
interest to the general public.
Director of
Public Affairs
Ongoing
A calendar of future appearances has not been
developed as the sensitive nature of the
Commission’s quasi-judicial role in the industry
can often necessitate last minute changes in
attendance due to a variety of potential conflicts.
The Director of Public Affairs will however
examine this proposal in the new fiscal year with
a view to providing as much information to the
public as possible.
Director of
Public Affairs
FY 2010/11
Recommendation #10
The Director General of Strategic Communications and Parliamentary Affairs, in
collaboration with the Secretary General and the Executive Directors of
Broadcasting, Telecommunications and PDR, should develop and implement an
integrated planning and reporting process and establish a pre-set forward agenda
to ensure that all major planning and reporting elements are reviewed at specified
times throughout the year.
Risk Type Audit Risk Rating Impact
CRTC Operational Audit 61
Operations Moderate Potential that senior management will not
maintain a strategic focus throughout the
year.
Response Office of Primary
Interest
Timeline
A CRTC Annual Corporate Planning and
Reporting cycle has been developed in
consultation with all sectors of the CRTC. It
will be presented for the approval of the
Operations Committee.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010
Development and approval of the cycle is the
first phase in establishing an integrated
planning and reporting process throughout the
organization. Additional processes will be
identified and incorporated into the process
over the next few months. (For example,
further input from members of the Corporate
Management Committee will be solicited in
J anuary 2010.)
Director General,
Strategic
Communications
and Parliamentary
Affairs
FY 2010/11
Recommendation #11
The Director General Finance and Administrative Services should continue to
reiterate to the Responsibility Centre Managers their responsibility for financial
budgeting and monitoring to ensure that they understand and provide complete,
accurate and timely financial information for the monthly budget summary report.
Risk Type Audit Risk Rating Impact
Monitoring Minor Potential that financial information
reported to senior management is not
complete and timely.
Response Office of Primary
Interest
Timeline
Procedures are in place whereby Finance staff
advises sector heads of the requirement to
review and update all their commitments
(Salary and O&M) on a monthly basis before
month end. In addition, as part of these
monthly notifications Finance staff have
advised sector heads that additional financial
training to sector management teams would
be provided upon request.
Director General,
Finance and
Administrative
Services
Ongoing
The requirement to do a comprehensive and
monthly review of all financial commitments
was recently re-emphasized by the Secretary
Secretary General November
2009
CRTC Operational Audit 62
General (Chief Financial Officer) to the
Executive Committee as part of the FY
2009/10 mid-year budget review.
Executive performance agreements have been
updated to reflect the expectation that all
members of the Executive Committee and
their management teams will review and
update their commitments monthly.
Director General,
Human Resources
Completed
Recommendation #12
The Champions, in collaboration with their project leads and the Director
General of Strategic Communication and Parliamentary Affairs, should ensure
that the Quarterly Reports on Future Direction are consistently published in a
complete and timely manner.
Risk Type Audit Risk Rating Impact
Reporting Minor Potential that incomplete reporting may
cause a decline in employee support for
the initiative.
Response Office of Primary
Interest
Timeframe
The Director General of Strategic
Communications and Parliamentary Affairs
will reiterate the importance of quarterly
reporting to the project leads and ensure that
all information is provided in a complete and
timely manner.
Director General,
Strategic
Communications
and Parliamentary
Affairs
J anuary
2010 and
ongoing
Publication dates for quarterly reports will be
pre-set and the Strategic Communications
sector will ensure that all required information
is submitted in conformity with the relevant
timelines.
Director General,
Strategic
Communications
and Parliamentary
Affairs
April 2010
and ongoing
Recommendation #13
The Future Direction Champion and Project Manager for Streamlined Rules and
Regulations, in collaboration with the Executive Directors should:
• Identify the activities that constitute the Future Direction
streamlining and policy direction initiatives underway;
• Identify possible activities that can be integrated;
• Update the activity catalogue on a regular basis; and,
CRTC Operational Audit 63
CRTC Operational Audit 64
• Clearly establish responsibility for the management of the key area
as well as other initiative and ensure that complete and timely bi-
weekly progress reporting is implemented.
Risk Type Audit Risk Rating Impact
Operations
Risk
Minor Potential that not all activities are
effectively managed.
Response Office of
Primary
Interest
Timeframe
Priorities for the Streamlined Rules and
Regulations initiative were initially established
through a Process Mapping exercise that
identified process “hot spots” most in need of
attention. Remaining activities will continue to
be reviewed and re-prioritized on an ongoing
basis according to operational requirements and
resources.
Future Direction
Champion,
Streamlined
Rules and
Regulations
Ongoing
Project management will collaborate with
Strategic Communications to ensure that
quarterly status updates are issued on a regular
and timely basis.
Project
Manager,
Streamlined
Rules and
Regulations
J anuary 2010
and ongoing
Project scope and management responsibility
will be further clarified through the upcoming
development of Future Direction Exercise -
Phase II.
Future Direction
Champion,
Streamlined
Rules and
Regulations
April 2010
doc_832403287.pdf