x

A PHP Error was encountered

Severity: Notice

Message: Undefined variable: content_category

Filename: user/transcript.php

Line Number: 106

A PHP Error was encountered

Severity: Warning

Message: Invalid argument supplied for foreach()

Filename: user/transcript.php

Line Number: 106

Behavioral Demand Letter Experiment

Total Views  :   611
Total Likes  :  0
Total Shares  :  0
Total Comments :  0
Total Downloads :  0

Add Comments
Presentation Slides

1) Behavioral Demand Letter Experiment Findings from a Randomized Controlled Trial Tammy Chang August 12, 2014

2) Background • Federal agencies refer delinquent debts to Treasury at 180 days past delinquency (batch/daily/etc referrals) • Treasury issues initial “demand letters” to debtors – Notice of Referral – Debt information – Repayment options • Debts serviced internally for 30 days Page 2 LEAD ∙ TRANSFORM ∙ DELIVER

3) Analytics Objectives • Design and run a rigorous and statistically sound experiment – Implement a Randomized Controlled Trial (RCT) • Develop and test an alternative letter – Can a letter incorporating insights from behavioral science change debtor behavior? Page 3 LEAD ∙ TRANSFORM ∙ DELIVER

4) Behavioral Science Concepts • • • • • • Social norm Limited attention Information overload Personalization Hassle factors Primacy Page 4 LEAD ∙ TRANSFORM ∙ DELIVER

5) Collaboration and Context • US Social and Behavioral Sciences Team – Identify opportunities for behavioral insights to increase efficiency and efficacy of federal agency programs • Evidence and Innovation Agenda (OMB M-13-17) – Promote evidence-based policy – Using data to inform and improve agency results – Rigorous, low-cost experiments • 2014 Economic Report of the President Page 5 LEAD ∙ TRANSFORM ∙ DELIVER

6) Outcome Measures of Interest • Collections – Likelihood of collection • Debtor contact – Likelihood of inbound calls • Payment method – Likelihood of self-initiated Pay.gov payment Page 6 LEAD ∙ TRANSFORM ∙ DELIVER

7) Demand Letter Examples Control Page 7 Treatment LEAD ∙ TRANSFORM ∙ DELIVER

8) Demand Letter Examples Limited Attention Treatment Social Norm Personalization Primacy Hassle Factor Previous Pay.gov link: https://www.pay.gov/paygov/forms/formIn stance.html?agencyFormId=16531440 New Pay.gov link: www.pay.gov/paygov/paymydebt Messenger Page 8 LEAD ∙ TRANSFORM ∙ DELIVER

9) Measuring Impact: Non-RCT Example • Before vs. After 5 4.5 4 Start of Pilot Percent 3.5 3 Impact? 2.5 2 1.5 1 0.5 0 Time 0 1 • What would have happened without the pilot? Page 9 LEAD ∙ TRANSFORM ∙ DELIVER

10) Measuring Impact: Example 1 • Control vs. Treatment 5 4.5 4 Treatment Start of Pilot 3.5 Percent 3 Impact 2.5 Control/Counterfactual 2 1.5 1 0.5 0 Page 10 Time 0 1 LEAD ∙ TRANSFORM ∙ DELIVER

11) Measuring Impact: Example 2 • Control vs. Treatment 5 Control/Counterfactual 4.5 4 Start of Pilot Treatment Impact 3.5 Percent 3 2.5 2 1.5 1 0.5 0 Page 11 Time 0 1 LEAD ∙ TRANSFORM ∙ DELIVER

12) Measuring Impact: Example 3 • Control vs. Treatment 5 4.5 4 Start of Pilot 3.5 Percent 3 2.5 2 1.5 Impact 1 0.5 0 Page 12 Treatment Control/Counterfactual Time 0 1 LEAD ∙ TRANSFORM ∙ DELIVER

13) RCT Methodology • Use of control/treatment groups • Random assignment – Last digit of Social Security Number (SSN) – SSN Structure XXX-XX-XXXX (prior to June 2011) Area number Group Number Serial Number (random) Even Digit: Control Group Current Letter XXX-XX-XXX0 XXX-XX-XXX2 XXX-XX-XXX4 XXX-XX-XXX6 XXX-XX-XXX8 Page 13 Odd Digit: Treatment Group New Letter XXX-XX-XXX1 XXX-XX-XXX3 XXX-XX-XXX5 XXX-XX-XXX7 XXX-XX-XXX9 LEAD ∙ TRANSFORM ∙ DELIVER

14) Sample • Six agency programs – Consistent referrals – Smaller balances – Administrative debts Agency Programs • VA: Compensation and Pension • Railroad Retirement Board: Unemployment Insurance Act Annuitant DOD: Defense Financial Accounting Services • • DOD: Brooke Army Medical Center • USDA: Administrative Consumer • Treasury: Securities and Savings Bonds • Referrals from October 28, 2013 – March 31, 2014 Page 14 LEAD ∙ TRANSFORM ∙ DELIVER

15) Outcome Measures Data • Existing data from FedDebt – – – – – Referral Dates Collection Dates and Type Call Dates and Types Payment Type Debt Characteristics • Timeframe – 30 days after referral – 60 days after referral Page 15 LEAD ∙ TRANSFORM ∙ DELIVER

16) Descriptive Statistics • Equal assignment across control/treatment Control Treatment Total % Control 1,270 1,264 2,534 50.12% 359 361 720 49.86% DFAS 8,246 8,282 16,528 49.89% BAMC 205 186 391 52.43% USDA 445 374 819 54.33% RRB 167 146 313 53.35% 10,692 10,613 21,305 50.19% VA BPD Total • Percent of debts with any collection (within 30 days of referral, by agency) Collection 167 No Collection 2,367 Total 2,534 % With Coll 6.59% BPD 92 628 720 12.78% DFAS 900 15,628 16,528 5.45% BAMC 7 384 391 1.79% USDA 65 754 819 7.94% RRB 23 290 313 7.35% Total 1,254 20,051 21,305 5.89% VA Page 16 LEAD ∙ TRANSFORM ∙ DELIVER

17) Results: Collections • Likelihood of Collection – Lower likelihood of collection on treatment group Percent of Debts with Collection 6% Collections within 30 days of referral Control No collection Total % With Coll Treatment Total 637 617 1,254 5.89% 10,055 9,996 20,051 94.11% 10,692 10,613 21,305 100% 50.19% Collection 49.81% 5.96% 5.81% Difference: -0.15% 5% 4% 3% 2% 1% 0% Control – Difference statistically insignificant Page 17 LEAD ∙ TRANSFORM ∙ DELIVER Treatment

18) Results: Calls • Likelihood of Inbound Call – Lower likelihood of calls from treatment group 14% Inbound calls within 30 days of referral Control Treatment Total Inbound call 1,473 1,396 2,869 13.47% No inbound call 9,219 9,217 18,436 86.53% 10,692 10,613 21,305 100% 13.78% 13.15% Difference: -0.62% Total % With Call Percent of Debts with Inbound Call Record 12% 10% 8% 6% 4% 2% 0% Control – Difference statistically insignificant Page 18 LEAD ∙ TRANSFORM ∙ DELIVER Treatment

19) Results: Pay.gov • Likelihood of Pay.gov payment – Higher likelihood of self-initiated Pay.gov Percent of Debts with payments from treatment group Pay.gov Payment 5% Payments within 30 days of referral Control Treatment Total 529 593 1,122 5.27% 10,163 10,020 20,183 94.73% Total 10,692 10,613 21,305 100% % Pay.gov 4.95% 5.59% Difference: 0.64% Pay.gov No or other payment – Odds ratio of 1.5: treatment group odds 1.5 times higher than control group odds of having Pay.gov payment 4% 3% 2% 1% 0% – Difference statistically significant Page 19 LEAD ∙ TRANSFORM ∙ DELIVER Control Treatment

20) Discussion/Recommendations • Findings on likelihood of collection depart from previous research • No effect on payment or contact likelihood – What does this tell us? • Ability to influence payment method decision – Change other Pay.gov references • Successful RCT! Page 20 LEAD ∙ TRANSFORM ∙ DELIVER

21) Contact Information Questions? Tammy Chang Sr Economist Debt Management Services Bureau of the Fiscal Service 202-874-5205 tammy.chang@fiscal.treasury.gov Page 21 LEAD ∙ TRANSFORM ∙ DELIVER