Creating some grins, and giggles with data. I have been playing around with our large Dow Jones text archive. All it took was a few Perl scripts, a spreadsheet and some time. I wrote a quick script just to look for word appearances out of a pool of articles from 1987 to 1994. The counts are based on a certain word appearing at least once in a story. Each success is then counted as a story hit. So all the counts are based on story hits. You can do a lot more, this is very basic but fun. Remember this is very raw and non-scientific but semi-interesting to say the least. I picked the following terms:
collapse
bankrupt
default
upgrade
downgrade
Year Collapse Bankrupt Earnings Default Upgrade Downgrade Total Articles
1987 155 651 5,261 253 391 483 68,430 1988 132 672 5,022 263 507 540 69,095 1989 177 1,065 7,434 399 644 825 83,934 1990 358 1,826 9,157 694 753 1,300 95,914 1991 492 2,124 10,932 747 1,197 1,526 114,861 1992 461 2,232 15,486 676 1,992 1,900 128,287 1993 451 1,810 18,103 528 2,738 2,022 165,601 1994 1,027 2,508 29,538 739 4,191 2,776 295,397
I also picked 'earnings' as a benchmark term. Why? I thought it might be a good word that appears enough times in a financial publication that would give me an idea of scale. In doing so I created a simple measure for myself to look at word frequency: (term appearing in a X stories / baseline term appearing in X stories) * 100. Example, 1987 "collapse" using "earnings" as a baseline term = (155/5261) * 100 = 2.95 vs. 1987 "bankrupt" using "earnings" as a baseline (651/5261) * 100 = 12.37. Below is a table with calculated results.
| Year | collapse | bankrupt | default | upgrade | downgrade |
|---|---|---|---|---|---|
| 1987 | 2.95 | 12.37 | 4.81 | 7.43 | 9.18 |
| 1988 | 2.63 | 13.38 | 5.24 | 10.10 | 10.75 |
| 1989 | 2.38 | 14.33 | 5.37 | 8.66 | 11.10 |
| 1990 | 3.91 | 19.94 | 7.58 | 8.22 | 14.20 |
| 1991 | 4.50 | 19.43 | 6.83 | 10.95 | 13.96 |
| 1992 | 2.98 | 14.41 | 4.37 | 12.86 | 12.27 |
| 1993 | 2.49 | 10.00 | 2.92 | 15.12 | 11.17 |
| 1994 | 3.48 | 8.49 | 2.50 | 14.19 | 9.40 |
1990 & 1991 were watershed years for the word "collapse" according to the index. In 1990 the appearances of the word "collapse" jumped from 177 stories to 358 stories, 102% increase and total headlines only increased from 1990 to 1991 by 20%; by that same measure my "earnings" benchmark only grew by 23%. Drexel Lambert "collapsed" in 1990 taking 39 stories in the count with it. There were 388 stories all together on Drexel in 1990. In 1991 "collapse" appeared in 492 stories overall. Remember The Gulf War was a big topic that year with 2,987 stories mentioning Iraq in some way but only 33 of those 2,987 mentioned the term "collapse". As a contrast Russia (remember the collapse of the Soviet Union) was mentioned in some way in 817 stories with 39 mentions of "collapse" and "soviet" appeared at least once in 3,045 stories and in those stories there were 104 that mentioned "collapse". Out of 492 "collapse" stories 9 mention "scandal". And out of the 492 stories mentioning "collapse", "real estate" is mentioned in 39 of them. 1994 would see a huge increase in the stories mentioning "collapse" with 1,027, a 127% increase from 1993. However our number of headlines grew by 78% and the "earnings" term mention grew by 68%. I'll do more digging on that at a later date. Here are some sample headlines from 1991:
- Bank Of New England Collapse May Put Focus On Regulators
- B.F. Goodrich CFO Collapses During Analyst Presentation
- Report Schwarzkopf Says Iraqi Army Near Collapse
- Some Iraq Units Said Close To Collapse; Some In Good Shape
- Iraqi Ridicules Schwarzkopf Comment Iraq Army May Collapse
- Investcorp Sold Intl Leisure Shrs Before Leisure Collapsed
- Vista Chemical Dn; Traders Cite Fear Of RWE Deal Collapse
- Riegle Seeks Probe Of Bank of New England Collapse
- Govt Allows Firm To Cover Losses From Exec Life Collapse
- Dam Collapses In Romania; At Least 66 Reported Dead
- Soviet Communist Party Near Financial Collapse Paper Says
|
|
|
|
Also another small anecdote, the following headline appeared on September 27, 1991
Wall Street Hired 6,000 In 2Q First Increase Since 1987
That headline coincides nicely with the beginning of an uptick in the word "upgrade". See the the chart above on terms, 1992 shows "upgrade" starting to increase relative to my other terms. Here is a chart of the Dow from 1987 - 1994 for comparison
|
|
|
|
| Year | No. of Filings |
|---|---|
| 1987 | 88,236 |
| 1988 | 68,482 |
| 1989 | 62,478 |
| 1990 | 64,688 |
| 1991 | 67,714 |
| 1992 | 71,725 |
| 1993 | 64,857 |
| 1994 | 54,425 |
Look for future updates to this idea as I get further into the data. Enjoy!
Comments