Results for *

Displaying results 1 to 10 of 10.

  1. Research, Ethics and Risk in the Authoritarian Field

    research ethics; authoritarian countries; research methodology; research in the authoritarian field; risk and field research; ethics and field research; authoritarianism; advice for field research; research transparency; research methods; ... more

     

    research ethics; authoritarian countries; research methodology; research in the authoritarian field; risk and field research; ethics and field research; authoritarianism; advice for field research; research transparency; research methods; mental impact of field research; planning for field research; dangers of field research

     

    Export to reference management software   RIS file
      BibTeX file
  2. A protocol for structured robustness reproductions and replicability assessments
    Published: 2024
    Publisher:  RWI - Leibniz-Institut für Wirtschaftsforschung, Essen, Germany

    Robustness reproductions and replicability discussions are on the rise in response to concerns about a potential credibility crisis in economics. This paper proposes a protocol to structure reproducibility and replicability assessments, with a focus... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    Resolving-System (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 10
    No inter-library loan

     

    Robustness reproductions and replicability discussions are on the rise in response to concerns about a potential credibility crisis in economics. This paper proposes a protocol to structure reproducibility and replicability assessments, with a focus on robustness. Starting with a computational reproduction upon data availability, the protocol encourages replicators to prespecify robustness tests, prior to implementing them. The protocol contains three different reporting tools to streamline the presentation of results. Beyond reproductions, our protocol assesses adherence to the pre-analysis plans in the replicated papers as well as external and construct validity. Our ambition is to put often controversial debates between replicators and replicated authors on a solid basis and contribute to an improved replication culture in economics. Die Diskussionen über die Robustheit von Reproduktionen und die Replizierbarkeit nehmen angesichts der Besorgnis über eine mögliche Glaubwürdigkeitskrise in den Wirtschaftswissenschaften zu. In diesem Papier wird ein Protokoll zur Strukturierung von Reproduzierbarkeits- und Replizierbarkeitsbewertungen vorgeschlagen, wobei der Schwerpunkt auf der Robustheit liegt. Das Protokoll beginnt mit einer rechnerischen Reproduktion bei Datenverfügbarkeit und ermutigt die Replikatoren, Robustheitstests vor ihrer Durchführung festzulegen. Das Protokoll enthält drei verschiedene Berichtswerkzeuge, um die Präsentation der Ergebnisse zu vereinfachen. Über die Reproduktionen hinaus bewertet unser Protokoll die Einhaltung der Prä-Analysepläne in den replizierten Arbeiten sowie die externe und konstruktive Validität. Unser Ziel ist es, die oft kontroversen Debatten zwischen Replikatoren und Replikationsautoren auf eine solide Basis zu stellen und zu einer verbesserten Replikationskultur in den Wirtschaftswissenschaften beizutragen.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Language: English
    Media type: Ebook
    Format: Online
    ISBN: 9783969732847
    Other identifier:
    hdl: 10419/306836
    Series: Ruhr economic papers ; #1106
    Subjects: Replication; reproducibility; robustness; research transparency; meta-science
    Scope: 1 Online-Ressource (circa 33 Seiten), Illustrationen
  3. Long-term effects of the targeting the ultra-poor program
    a reproducibility and replicability assessment of Banerjee et al. (2021)
    Published: 2024
    Publisher:  RWI - Leibniz-Institut für Wirtschaftsforschung, Essen, Germany

    Banerjee, Duflo, and Sharma (BDS, 2021a) conduct a ten-year follow-up of a randomized transfer program in West Bengal. BDS find large effects on consumption, food security, income, and health. We conduct a replicability assessment. First, we... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    Resolving-System (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 10
    No inter-library loan

     

    Banerjee, Duflo, and Sharma (BDS, 2021a) conduct a ten-year follow-up of a randomized transfer program in West Bengal. BDS find large effects on consumption, food security, income, and health. We conduct a replicability assessment. First, we successfully reproduce the results, thanks to a perfectly documented reproduction package. Results are robust across alternative specifications. We furthermore assess the paper’s pre-specification diligence and the reporting in terms of extern. Banerjee, Duflo und Sharma (BDS, 2021a) führen eine zehnjährige Nachuntersuchung eines randomisierten Transferprogramms in Westbengalen durch. BDS finden große Auswirkungen auf Konsum, Ernährungssicherheit, Einkommen und Gesundheit. Wir führen eine Bewertung der Replizierbarkeit durch. Zunächst gelingt es uns, die Ergebnisse dank eines perfekt dokumentierten Reproduktionspakets zu reproduzieren. Die Ergebnisse sind robust gegenüber alternativen Spezifikationen. Darüber hinaus bewerten wir die Sorgfalt, mit der die Studie vor der Spezifikation erstellt wurde, und die Berichterstattung in Bezug auf die externen Daten.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Contributor: Banerjee, Abhijit V. (VerfasserIn des Bezugswerks); Duflo, Esther (VerfasserIn des Bezugswerks); Sharma, Garima (VerfasserIn des Bezugswerks)
    Language: English
    Media type: Ebook
    Format: Online
    ISBN: 9783969732854
    Other identifier:
    hdl: 10419/306837
    Series: Ruhr economic papers ; #1107
    Subjects: Replicability; randomized controlled trial; transfer programs; research transparency
    Scope: 1 Online-Ressource (circa 33 Seiten), Illustrationen
  4. Long-term effects of the targeting the ultra-poor program
    a reproducibility and replicability assessment of Banerjee et al. (2021)
    Published: August 2024
    Publisher:  Institute for Replication, Essen, Germany

    Banerjee, Duflo, and Sharma (BDS, 2021a) conduct a ten-year follow-up of a randomized transfer program in West Bengal. BDS find large effects on consumption, food security, income, and health. We conduct a replicability assessment. First, we... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    Verlag (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 831
    No inter-library loan

     

    Banerjee, Duflo, and Sharma (BDS, 2021a) conduct a ten-year follow-up of a randomized transfer program in West Bengal. BDS find large effects on consumption, food security, income, and health. We conduct a replicability assessment. First, we successfully reproduce the results, thanks to a perfectly documented reproduction package. Results are robust across alternative specifications. We furthermore assess the paper's pre-specification diligence and the reporting in terms of external and construct validity. While the paper refers to a pre-registration, it lacks a pre-analysis plan. Assessing the validity of findings for other contexts is difficult absent necessary details about the exact treatment delivery.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Contributor: Banerjee, Abhijit V. (VerfasserIn des Bezugswerks); Duflo, Esther (VerfasserIn des Bezugswerks); Sharma, Garima (VerfasserIn des Bezugswerks)
    Language: English
    Media type: Book
    Format: Online
    Other identifier:
    hdl: 10419/301431
    Series: I4R discussion paper series / Institute for Replication ; no. 142
    Subjects: replicability; randomized controlled trial; transfer programs; research transparency
    Scope: 1 Online-Ressource (circa 32 Seiten), Illustrationen
  5. A protocol for structured robustness reproductions and replicability assessments
    Published: August 2024
    Publisher:  Institute for Replication, Essen, Germany

    Robustness reproductions and replicability discussions are on the rise in response to concerns about a potential credibility crisis in economics. This paper proposes a protocol to structure reproducibility and replicability assessments, with a focus... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    Verlag (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 831
    No inter-library loan

     

    Robustness reproductions and replicability discussions are on the rise in response to concerns about a potential credibility crisis in economics. This paper proposes a protocol to structure reproducibility and replicability assessments, with a focus on robustness. Starting with a computational reproduction upon data availability, the protocol encourages replicators to prespecify robustness tests, prior to implementing them. The protocol contains three different reporting tools to streamline the presentation of results. Beyond reproductions, our protocol assesses adherence to the pre-analysis plans in the replicated papers as well as external and construct validity. Our ambition is to put often controversial debates between replicators and replicated authors on a solid basis and contribute to an improved replication culture in economics.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Language: English
    Media type: Book
    Format: Online
    Other identifier:
    hdl: 10419/301917
    Series: I4R discussion paper series / Institute for Replication ; no. 143
    Subjects: replication; reproducibility; robustness; research transparency; meta-science
    Scope: 1 Online-Ressource (circa 34 Seiten), Illustrationen
  6. Effects of rural electrification on employment
    a comment on Dinkelman (2011)
    Published: February 2020
    Publisher:  RWI - Leibniz-Institut für Wirtschaftsforschung, Essen, Germany

    This paper replicates and extends the seminal paper by Dinkelman (2011) on the impacts of electrification on female employment. We revisit the validity of the identification strategy that uses the land gradient as an instrumental variable (IV). Our... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    Resolving-System (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 10
    No inter-library loan

     

    This paper replicates and extends the seminal paper by Dinkelman (2011) on the impacts of electrification on female employment. We revisit the validity of the identification strategy that uses the land gradient as an instrumental variable (IV). Our robustness checks cast doubt on the exclusion restriction as the IV drives the outcome variable in non-electrified regions. We also demonstrate that it is more difficult to disentangle the effects of electricity and road infrastructure than the original paper claims, because the IV affects both. We additionally highlight that the IV is weak, consequently preventing interpretation of the point estimates that are used throughout the original paper. The concomitance of a questionable exclusion restriction and a weak IV is particularly problematic. We conclude by arguing that the takeaways of the original paper for policy and the academic literature need to be reconsidered. In general terms, our comment shows the difficulties of using geographical variation as a natural experiment for infrastructure evaluation.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Language: English
    Media type: Ebook
    Format: Online
    ISBN: 9783867889735
    Other identifier:
    hdl: 10419/214184
    Series: Ruhr economic papers ; #840
    Subjects: Replication; research transparency; energy access; infrastructure; instrumental variables
    Scope: 1 Online-Ressource (circa 31 Seiten), Illustrationen
  7. Who nudges whom?
    field experiments with public partners
    Published: April 2021
    Publisher:  Ruhr-Universität Bochum (RUB), Department of Economics, Bochum, Germany

    Field experiments which test the application of behavioural insights to policy design have become popular to inform policy decisions. This study is the first to empirically examine who and what drives these experiments with public partners. Through a... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    Resolving-System (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 10
    No inter-library loan

     

    Field experiments which test the application of behavioural insights to policy design have become popular to inform policy decisions. This study is the first to empirically examine who and what drives these experiments with public partners. Through a mixed-methods approach, based on a novel dataset of insights from academic researchers, behavioural insight team members, and public servants, I derive three main results: Firstly, public servants have a considerable influence on study setup and sample design. Secondly, behavioural insight team members report concerns regarding scientific rigor and limitations imposed by risk-aversion of their public partners significantly more often than academic researchers. Thirdly, transparency and quality control in collaborative research are low with respect to pre-analysis plans, the publication of results, and medium or long term effects. To remedy the current weaknesses, the study sketches out several promising ways forward, such as setting up a matchmaking platform for researchers and public bodies to facilitate cooperation, and using time-embargoed pre-analysis plans. Feldexperimente, welche die Anwendung von verhaltenswissenschaftlichen Erkenntnissen auf Politikvorhaben testen, werden immer beliebter. Die vorliegende Studie untersucht international als erste empirische Arbeit die dahinterliegenden Kooperationen zwischen einem öffentlichen Partner auf der einen und entweder akademischen Forschern oder Mitarbeitern von Behavioural Insight Teams auf der anderen Seite. Mit neu erhobenen Daten von akademischen Forschern, Mitgliedern von Behavioural Insight Teams und öffentlichen Bediensteten leite ich unter Anwendung qualitativer und quantitativer Methoden drei Hauptergebnisse ab: 1. In kooperativen Experimenten hat die beteiligte öffentliche Institution einen entscheidenden Einfluss auf den Studienaufbau und das Stichprobendesign. 2. In der Zusammenarbeit mit einem öffentlichen Partner sehen sich Mitglieder von Behavioural Insight Teams deutlich häufiger durch die Risikoaversion ihres Partners in ihren wissenschaftlichen Möglichkeiten eingeschränkt als akademische Forscher. 3. Transparenz und Qualitätskontrolle sind in kollaborativen Experimenten mit öffentlichen Partnern gering, gemessen daran, wie häufig präspezifizierte Analyseplänen zum Einsatz kommen, die Ergebnisse der Experimente in irgendeiner Form veröffentlicht werden und Mittel- und Langfristeffekte erhoben werden, um diese mit den Kurzfristeffekten zu vergleichen. Die Studie skizziert einige vielversprechende Wege, um die aktuellen Schwachstellen zu adressieren, z.B. die Einrichtung einer Matchmaking-Plattform für Forscher und öffentliche Einrichtungen oder die Nutzung von präspezifizierten Analyseplänen mit Sperrfrist für die Veröffentlichung.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Language: English
    Media type: Ebook
    Format: Online
    ISBN: 9783969730492
    Other identifier:
    hdl: 10419/233882
    Series: Ruhr economic papers ; #906
    Subjects: Behavioural public policy; field experiments; Behavioural Insights Team (BIT); research transparency; expert interviews
    Scope: 1 Online-Ressource (circa 53 Seiten), Illustrationen
  8. Do economists replicate?
    Published: 2022
    Publisher:  RWI - Leibniz-Institut für Wirtschaftsforschung, Essen, Germany

    Reanalyses of empirical studies and replications in new contexts are important for scientific progress. Journals in economics increasingly require authors to provide data and code alongside published papers, but how much does the economics profession... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    Resolving-System (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 10
    No inter-library loan

     

    Reanalyses of empirical studies and replications in new contexts are important for scientific progress. Journals in economics increasingly require authors to provide data and code alongside published papers, but how much does the economics profession indeed replicate? This paper summarizes existing replication definitions and reviews how much economists replicate other scholars’ work. We argue that in order to counter incentive problems potentially leading to a replication crisis, replications in the spirit of Merton’s ‘organized skepticism’ are needed – what we call ‘policing replications’. We review leading economics journals to show that policing replications are rare and conclude that more incentives to replicate are needed to reap the fruits of rising transparency standards.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Language: English
    Media type: Ebook
    Format: Online
    ISBN: 9783969731000
    Other identifier:
    hdl: 10419/250076
    Series: Ruhr economic papers ; #939
    Subjects: Replication; research transparency; generalizability
    Scope: 1 Online-Ressource (circa 23 Seiten), Illustrationen
  9. Do economists replicate?
    Published: January 2023
    Publisher:  Institute for Replication, Essen, Germany

    Reanalyses of empirical studies and replications in new contexts are important for scientific progress. Journals in economics increasingly require authors to provide data and code alongside published papers, but how much does the economics profession... more

    Access:
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 831
    No inter-library loan

     

    Reanalyses of empirical studies and replications in new contexts are important for scientific progress. Journals in economics increasingly require authors to provide data and code alongside published papers, but how much does the economics profession actually replicate? This paper summarizes existing replication definitions and reviews how much economists replicate other scholars' work. We argue that in order to counter incentive problems potentially leading to a replication crisis, replications in the spirit of Merton's 'organized skepticism' are needed - what we call 'policing replications'. We review leading economics journals to show that policing replications are rare and conclude that more incentives to replicate are needed to reap the fruits of rising transparency standards.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Language: English
    Media type: Book
    Format: Online
    Other identifier:
    hdl: 10419/267931
    Series: I4R discussion paper series / Institute for Replication ; no. 13
    Subjects: replication; replicability; research transparency; metascience; generalizability; systematic review
    Scope: 1 Online-Ressource (circa 30 Seiten), Illustrationen
  10. Mass reproducibility and replicability
    a new hope
    Published: April 2024
    Publisher:  IZA - Institute of Labor Economics, Bonn, Germany

    This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness... more

    Access:
    Verlag (kostenfrei)
    Verlag (kostenfrei)
    Resolving-System (kostenfrei)
    ZBW - Leibniz-Informationszentrum Wirtschaft, Standort Kiel
    DS 4
    No inter-library loan

     

    This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we uncover a high rate of fully computationally reproducible results (over 85%). Second, excluding minor issues like missing packages or broken pathways, we uncover coding errors for about 25% of studies, with some studies containing multiple errors. Third, we test the robustness of the results to 5,511 re-analyses. We find a robustness reproducibility of about 70%. Robustness reproducibility rates are relatively higher for re-analyses that introduce new data and lower for re-analyses that change the sample or the definition of the dependent variable. Fourth, 52% of re-analysis effect size estimates are smaller than the original published estimates and the average statistical significance of a re-analysis is 77% of the original. Lastly, we rely on six teams of researchers working independently to answer eight additional research questions on the determinants of robustness reproducibility. Most teams find a negative relationship between replicators' experience and reproducibility, while finding no relationship between reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.

     

    Export to reference management software   RIS file
      BibTeX file
    Source: Union catalogues
    Language: English
    Media type: Book
    Format: Online
    Other identifier:
    hdl: 10419/295935
    Series: Discussion paper series / IZA ; no. 16912
    Subjects: reproduction; replication; research transparency; open science; economics; political science
    Scope: 1 Online-Ressource (circa 246 Seiten), Illustrationen