Atlas Framework for Analyzing Curricular ContentThe Atlas framework employs the content categories to map the curricular content of courses offered in MPP/MPA programs and the advice offered by IGOs. Using the methodology described below, entries in seven of these categories - teaching topics, advisory topics, concepts, courses, IGO reports, competencies, and reference collections - are assigned to the 33 subjects in the four domains shown on the Subjects page.
How the field of public policy and management is modeled and mapped to facilitate comparison of online content
The Content Categories in the Atlas Framework
- Domain: a broad area of public policy and management characterized by the nature of pedagogical preparation or of government practice. There are two general preparation domains: Analysis and Skills, where subjects and topics tend to have a "how to" orientation; and Institutions and Context, where subjects and topics tend to have a "why does" orientation. And there are two specific practice domains: Management Functions, where subjects and topics are associated primarily with organizations responsible for management practices; and Policy Sectors, where subjects and topics are associated primarily with sectoral ministries responsible for policy execution. University teaching tends to be concentrated on the two general preparation domains. Most university programs also offer courses in the specific practice domains, with MPP programs typically having more courses in Policy Sectors subjects than MPA programs, and MPA programs typically having more courses in Management Functions subjects than MPP programs. Advice from IGOs is concentrated on subjects in the two specific practice domains.
- Subject: a collectively exhaustive set of mutually exclusive topics in public policy and management (see below for elaboration of these terms). The names we have selected for subjects are derived from names commonly associated with a single university course and/or a government ministry. The Atlas uses 33 subjects, each assigned to one of the Atlas's four domains. For example, Quantitative Methods is one of the 6 subjects in the Analysis and Skills domain; Democratic Institutions and Policy Process is one of the 4 subjects in the Institutions and Context domain; Public Financial Management is one of the 7 subjects in the Management Functions domain and International Development is one of the 16 subjects in the Policy Sectors domain. An Atlas subject differs from an “academic subject” (for example, macroeconomics or international relations) which is not intended to be a mutually exclusive category. Whereas one academic subject can share topics with another (for example, competitive devaluation is a topic in both macroeconomics and international relations), an Atlas subject is a unique set of mutually exclusive topics. Atlas subjects are thus mutually exclusive in that they do not overlap with other Atlas subjects in their topic content. As described on the Subjects page, subjects can also be classified as Policy-Oriented or Management-Oriented. Some can also be classified as having high Math-Econ (mathematics and economics) content. These distinctions are used in the MPP/MPA Curricular Types categorization. And some can be classified as Archetypal Public Affairs while some others are Archetypal International Affairs. These distinctions are used in Comparisons with International Affairs Programs.
- MPP/MPA Program: a program of study leading to a Master of Public Policy, Master of Public Administration, or similar degree offered by a university. In most cases the programs are provided through a specific institution (usually called a School) within the university. The institutions with one or more programs included for analysis on the Atlas are listed at MPP/MPA Programs. For each of these institutions, there is a program page (see Harvard Kennedy School for an example) that provides key program details in a common format. The courses offered are arrayed on an institutional course map indicating the subject most associated with the course content and from this an instructional distribution profile is calculated (see PEACO Algorithm). These profiles enable comparisons among institutions on matters such as the distribution of instruction among Atlas subject-matter domains or NASPAA competency domains (see Curriculum Comparison Tables).
- MPP/MPA Course: a university offering that teaches an aspect of public policy and management. Most courses on the Atlas are at the Master’s level and are one semester (typically 3 hours per week of instruction over 12-14 weeks). Although most courses are not exclusive to a single public policy and management subject, many can be primarily associated with a particular subject. For example, University of Toronto’s PPG-2011: Ethics and the Public Interest is primarily associated with the institutions and context subject of Ethics, Rights and Accountability, and Harvard University’s API-101: Markets and Market Failure is primarily associated with the analysis and skills subject of Economic Analysis. Although the Atlas lists virtually all the courses offered by each institution on the institutional course maps on their respective program pages (approximately 3,000 courses in total), we are using a more limited sample of about 130 courses displayed on Illustrative Courses Map to identify the principal teaching topics in the field of public policy and management. These courses are selected from the over 600 courses offered by eight institutions: Harvard Kennedy School (with 225 courses, the most comprehensive of the American programs), NYU Wagner (with about 135 courses) and six Canadian schools (Carleton SPPA, Dalhousie SPA, Ottawa GSPIA, Queen's SPS, Saskatchewan-Regina JSGS, and Toronto SPPG). Each of these institutions has made most of its course syllabi available online, either through its institutional websites or through the Atlas.
- Teaching Topic: a coherent body of concepts and readings suitable for a single 3-hour class in an MPP or MPA program. Each teaching topic is assigned exclusively to one subject. For example, Cost-Benefit Analysis is a teaching topic assigned to Analytic Methods and Consumer Theory is a teaching topic assigned to Economic Analysis. The names of many of the Atlas teaching topics are derived from the thematic titles used for individual classes in a course. The procedure for generating teaching topics involves four steps: 1) list on each Atlas course page the titles used in the syllabus to describe the subject matter taught in each class; 2) drawing on these class titles, and taking account of those in similar courses, create a list of teaching topics and assign each one to its appropriate subject; 3) create an Atlas page for each teaching topics and populate it with relevant material from the syllabi; and 4) provide on each course page links organized by subject to the topics addressed in the course. (See Harvard MLD-101 for an example.) Once a teaching topic is specified and assigned, a link to it is created in Teaching Topics Map, where teaching topics are arranged by subject and in Topic Encylopedia, where teaching and advisory topics are arranged alphabetically. As noted in Normed Core Topics, our aim is to construct teaching topics on the Atlas so they have the properties of normed topics.
- Advisory Topic: a coherent body of practice advice and explanatory material provided to government in a report from an international agency. Each advisory topic is assigned exclusively to one subject. For example, Fiscal Rules is an advisory topic assigned to Public Financial Management. The names of many of the advisory topics are derived from the titles of reports or studies. These reports tend to have considerably more detail than course syllabi and the task of generating advisory topics from them is one of summarizing key points in a concise format. We use a common format that includes Summary Advice, Main Points, and Commentary. For advisory topics drawing on reports from more than one agency, the Summary Advice and Main points from each agency are labelled. (See Comprehensive Government Budgets (OECD and IMF) for an example.) Once an advisory topic is specified and assigned, a link to it is created in Practice Advice Map, where advisory topics are arranged by subject and in Topic Encylopedia, where teaching and advisory topics are arranged alphabetically. Our aim is to construct advisory topics on the Atlas so they, like teaching topics, have the properties of normed topics.
- Concept: an abstract idea from a theory or from a generalization of particular instances, or a definition of a term. Concepts (and terms) are employed in all teaching topics and courses, and in all practice advice. The method used to generate the collection of concepts differs from that used to generate the collections of topics in that it did not rely primarily on online sources. Most of the concepts and terms were identified by student research assistants actually taking MPP/MPA courses at the six Canadian programs noted above (Carleton SPPA, Dalhousie SPA, Ottawa GSPIA, Queen's SPS, Saskatchewan-Regina JSGS, and Toronto SPPG) in the period 2008-2011. Sources include class notes as well as published syllabi. The approximately 1,000 concepts and terms in the Atlas collection are arrayed alphabetically in Concept Dictionary and by the subject they are most closely associated in Concept Map. Although some concepts are exclusively associated with a particular topic and subject, others are not. Unlike topics, in the Atlas concepts are not exclusive to any particular topic or subject. For example, Accrual Basis of Accounting is almost exclusively associated with Public Financial Management but Blame Avoidance, while being primarily associated with the Ethics, Rights and Accountability, it is also associated with several other subjects.
- International Governmental Organization (IGO): an international organization (sometimes called intergovernmental organization) that provides advice to member governments. The four major IGOs providing advice on public policy and management are the Organisation for Economic Cooperation and Development (OECD), the World Bank, the United Nations Development Programme (UNDP) and the International Monetary Fund (IMF). In addition, there are a number of regional and specialized agencies that provide advice to governments. (See IGOs.) We have mapped a substantial amount of practice advice from the OECD on the OECD Map at Organisation for Economic Cooperation and Development.
- IGO Report: a report published by an international governmental organization (IGO), usually following a study or a review of a topic and/or a country. For convenience and to allow word searching, the online reports are usually downloaded and a page created for them in the Atlas. Links to the 60 reports that currently have their own Atlas pages can be found on IGO Reports, where the links are organized by IGO, and on IGO Reports Map, where they are organized by the primary subject they address.
- Reference Collection: a group of bibliographic references associated with a topic. The Atlas database includes about 3,300 references derived from assigned course readings and student papers submitted in the six Canadian programs noted above. RefWorks software was used to organize references by class topic or student paper topic as individual RefShare folders. The resulting collections are organized by subject on References Map. The "class topic" and "student paper topic" used to organize the reference collections were not identical to the teaching topics that have subsequently been developed in the Atlas analysis and displayed in Teaching Topics Map.
- Competency: a technical or professional skill or a personal attribute or underlining characteristic that enables the delivery of a role or job. We have derived a set of MPP/MPA Core Competencies based on our analysis of the content of required courses in well-established programs and the competency standards of the Network of Schools of Public Policy, Affairs and Administration (NASPAA, NASPAA, 2012, pp 69-71) and the United Nations Department of Economic and Social Affairs/International Association of Schools and Institutes of Administration Task Force on Standards of Excellence for Public Administration Education and Training (UNDESA/IASIA, 2008, pp 9-10). We also provide the competency lists from one government (Treasury Board of Canada Secretariat, Key Leadership Competencies) and one international agency (OECD Core Competencies). The competencies provided by the first three are mapped to public policy and management subjects on Competencies Map.
The linkages to maps for these content categories are brought together in All Categories Map.
POTENTIAL ADDITIONAL CONTENT CATEGORIES
- Academic Paper: a scholarly document presented at a conference or published in a journal that addresses a theoretical or empirical aspect of policy analysis, public policy, public administration or public management.
- Research Topic: a coherent body of concepts, argument and references suitable for presentation at a scholarly conference or publication in a scholarly journal. Similarly to teaching topics and advisory topics, each research topic is assigned exclusively to a single subject. A methodology for identifying one or more research topics from titles of papers, or sections of papers, would be similar to the process of identifying teaching topics from class titles in course syllabi and that of identifying advisory topics from agency reports. The first stage of the process (collecting titles) has been undertaken for the scholarly papers presented at the First International Conference on Public Policy (ICPP) held in Grenoble, June 26-28, 2013 (see ICPP Research Map) and for papers presented the First (2012) and Second (2013) Annual Conferences of CAPPA, the Canadian Association of Programs on Public Administration (see CAPPA Research Map).
Curricular Attributes for Comparing MPP/MPA Programs
The Atlas methodology described above generates information on MPP/MPA programs in a common format on a number of general attributes: geographic location, name of degree and specializations, minimum time to completion, marketing approach, academic unit within the university, tuition, enrolment, internships and co-curricular activities. It also provides data on quantifiable attributes of curriculum content available from university calendars, such as number of courses needed to graduate, the number of required and elective courses offered, and the subject (and therefore domain) to which courses are assigned by the Atlas editors.
This information is presented on the MPP/MPA Programs pages (including the course maps on these pages) and the Curriculum Comparison Tables to allow Atlas viewers to make comparisons on a program-to-program among basis among institutions. For the approximately 130 courses included on Illustrative Courses Map, viewers can also make comparisons on a course-to-course basis for each subject.
Exclusivity and Exhaustively
Some of the content categories are defined so that their entries are mutually exclusive; some are defined so that their entries can be collectively exhaustive (see illustration below). In particular:
- topics (both teaching and advisory) are intended to be mutually exclusive
- subjects are defined to be collectively exhaustive sets of topics
- subjects are mutually exclusive since they contain different sets of mutually exclusive topics
- domains are composed of collectively exhaustive sets of subjects (and, therefore, of collectively exhaustive sets of mutually exclusive topics)
- domains are mutually exclusive since each comprises a different set of mutually exclusive subjects (and topics)
It is worth noting that domain character deals with two levels of distinctions that we believe are important in characterizing pedagogy in this professional field. At the top level the distinction is whether the primary purpose of teaching the topic is that of general preparation for application to multiple areas of practice or whether it is that of preparing for a specific practice area. For the first, a further distinction is whether the focus is on developing tools and skills for application to practice, or whether it is on improving the understanding of the institutions and context relevant to practice. For the second, a further distinction is whether the practice area is one of the generic management functions or one of the policy sectors. Because the four domains noted above are the product of two levels of distinction (in contrast, for example, to the two-by-two matrix associated with binary distinctions at a single level), it would be theoretically possible to generate more than four domains. For example, the tools/context distinction could be applied within both the management functions domain and the policy sectors domain to create a total of six domains. In order to retain four mutually exclusive domains into which to assign topics and subjects the attribute distinctions have to be applied in a consistent order: the general/specific distinction applied before the tools/context distinction.
As in any classification system, choices have to be made and classification judgements applied where assignments to categories are not immediately obvious. The conventions that guide our classification judgements are outlined below.
Classification judgements are required to deal with the horizontal distinctions between domains (and therefore between subjects). The two general preparation domains (Analysis and Skills, Institutions and Context) are intended to contain subjects that provide students with theoretical constructs and knowledge that will be valuable to understand and address a range of public policy and management issues. The two specific practice domains (Management Functions, Policy Sectors) contain subjects of professional work in public policy and management. This gives rise to the question of whether a topic (such as Cost-benefit Analysis) that prepares students with a general technique that has special applicability to a particular practice area should be classified under the general preparation domain subject (in this case, Analytic Methods or under the specific practice domain subject (in this case, Evaluation and Performance Measurement). Similar questions arise for the classification of certain courses, concepts and study reports. We have adopted the following classification convention: where such classification choices arise for teaching topics, courses or concepts, we place them in the general preparation domain subjects (in this case, Analytic Methods rather than Evaluation and Performance Measurement); where they arise for advisory topics or study reports, we place them in the specific practice domain subjects.
Other classification questions arise from the distinction between the two general preparation domains: Analysis and Skills ("how to") and Institutions and Context ("why does"). The subject pair in which this question arises most frequently is Policy and Management Analysis (Analysis and Skills domain) and Democratic Institutions and Policy Process (Institutions and Context domain). We have found it helpful to imagine the pedagogical challenge or question being addressed in the subject matter. We have adopted the following classification convention: where the implicit challenge is "how to improve public policy and management" the topic is assigned to a subject in the Analysis and Skills domain; where the implicit question is "why does this occur?" it is assigned to a subject in the Institutions and Context domain.
There are vertical distinctions between topics. Given the way that Atlas teaching topics are defined, teaching topics derived from more specialized (usually elective) courses could be viewed as being situated "below" teaching topics in more general (often required) courses. For example, Foundations of Game Theory is a teaching topic in a general course in some programs and the Harvard Kennedy School offers a specialized course entitled Game Theory and Strategic Decisions, with teaching topics such as Mixed Strategies and Prisoner's Dilemma. We have opted not to create two levels of topic (such as topic/subtopic) to capture this vertical distinction; we call both Prisoner's Dilemma and Foundations of Game Theory teaching topics. However, in cases where the subtopics are not assigned to the same subject, given our convention that a teaching topic must be assigned exclusively to one subject, where such fine distinctions are needed, we call the higher level topic a "teaching topic grouping." For example, Public Service Reform is a class title in several general courses and the Queen's School of Policy Studies offers an elective course entitled Public Service Reform, with teaching topics such as Regulatory Reform, Contracting Out, and Performance Measurement. Because these latter topics are assigned to different subjects we call Public Service Reform a teaching topic grouping. We list it in the Topic Encyclopedia but do not assign it to any single subject in Topic Map. The same phenomenon can be found with advisory topics. The distinction between a topic and a topic grouping is unimportant for most purposes (such as developing or revising content for a particular course); it is only relevant for the epistemological purpose of defining a particular subject through its topics.
As a final point on the vertical distinctions issue, it is worth noting that concepts are not necessarily situated "below" topics. Although it is true any single teaching topic and or advisory topic is likely to draw on several concepts, it is also possible for a single concept (e.g., Democracy) to be the subject matter of several topics. Because topics and concepts are different in the way they are defined and used, there is no taxonomic inconsistency with having a topic and a concept use the same name. In practice, topic names on the Atlas tend to be longer than concept names, in the same way that encyclopedia entries tend to have longer names than dictionary entries.
Classification judgements are required to deal with non-exclusivity, particularly for courses, concepts and study reports. As noted above, the only content categories that are intended to be exclusively assigned to subjects are topics, both teaching topics and advisory topics. Other content categories are not intended to be exclusive: many courses and study reports address a variety of topics, not all of which are assigned to the same Atlas subject. Similarly, as already noted, many concepts are used in more than one topic and often in more than one subject. Although courses, concepts and study reports are assigned on their respective maps (Illustrative Courses Map, Concepts Map and IGO Reports Map) to a single subject (the one with which they are predominantly associated), their individual Atlas pages make reference to the other subjects to which they relate. For example, Harvard's MLD-101: Management, Leadership and Decision Making is assigned to Policy and Management Analysis on course maps because more of its teaching topics are in that subject than in others, but its course page notes the various subjects to which the course's teaching topics are assigned (11 to Policy and Management Analysis, 3 to Leadership Skills, and 2 to Evaluation and Performance Measurement).
Issues and Challenges
The definitions and names of Atlas subjects have evolved as we learned more about the curricular content of MPP and MPA programs around the world and as we learned more about the content of the advice provided by international agencies. For example, at one time we had a subject under the Management Functions domain named "Oversight, Risk Management and Control" which included topics and concepts related to audit, control and risk management but we found that there were few courses taught in this area and little international agency advice provided. We therefore eliminated this subject and expanded the definition of other subjects to include the topic formerly assigned to it. We assigned the audit-related topics and concepts to Evaluation and Performance Measurement and the control and risk management topics and concepts to Public Financial Management.
Some of the issues and challenges presented by the Atlas framework and classification conventions are noted below.
- Most of the subject matter taught in the macroeconomics subdiscipline of economics is assigned to Macroeconomic Policy (in the Policy Sectors domain) in contrast to the subject matter taught in microeconomics which is assigned to Economic Analysis (in the Analysis and Skills domain). This is because concepts in microeconomics (especially if they are taught with this in mind) provide powerful tools for analyzing a wide variety of public policy and management issues while the application of macroeconomics concepts is mostly confined to fiscal and monetary policy.
- Most of the subject matter taught in the policy-related subdisciplines of political science (policy analysis, policy studies, policy design, etc.) as well as the management-related subdisciplines (public management, strategic management, etc.) are assigned to Policy and Management Analysis (and a minority to Analytic Methods, also in the Analysis and Skills domain) even though it could be argued that in many political science departments this subject matter is taught with more of a “why does” than a “how to” orientation. We have done this on the assumption that in MPP and MPA programs, an effort is made to teach these subdisciplines with a "how to" orientation.
- Most of the subject matter taught in courses with titles such as “Qualitative Research Methods” is assigned to Analytic Methods. We have done this because there do not appear to be enough courses teaching this material to warrant a separate subject alongside Quantitative Methods, and because such courses do not generally require the same preparation in mathematics and economics as courses assigned to Quantitative Methods.
- Much of the subject matter that we have assigned to Leadership Skills could equally have been assigned to Policy and Management Analysis Indeed, many courses that deal with policy and management analysis address leadership as one topic among many. We have done this partly to balance subject matter among the subjects in the Analysis and Skills domain and partly out of the conviction that teachable leadership skills are important enough in the preparation of MPP and MPA graduates to warrant a separate subject.
- Some of the subject matter in Ethics, Rights and Accountability (especially that related to accountability) could equally be assigned to Democratic Institutions and Policy Process. We have done this partly to balance the subjects in the Institutions and Context domain.
- Two subjects, Civil Society and Advocacy, and Local Government Management, differ from other subjects in the Management Functions domain in that they are focused on an institutional sector rather than a function. These subject categories are intended to group material pertaining to these sectors in one place, even though it would be possible to assign some of these topics to subjects in the Analysis and Skills domain or the Institutions and Context domain.
Although classification challenges remain, it is worth remembering that for most purposes the crucial unit of public policy and management content is not the domain or subject, but the teaching or advisory topic. The important pedagogical question is not whether the topic is classified in a "how to" or a "why does" subject, but whether and how the topic is taught. To be properly prepared for the professional world, MPP and MPA students need a solid grasp of both the "how to" and "why does" aspects of public policy and management. Similarly, the important advisory question is not how the advice is classified but whether the advisory topic is pertinent and whether the specific advice is useful.
Page Prepared by: Ian Clark, last updated 2 June 2015.