I was attempting to collect information on U.S. Federal Employees and their Salaries and came across a few sites that allowed you to search and see a few salaries at once, but did not provide the entire raw data set.
examples:
http://www.fedsdatacenter.com/federal-pay-rates/
https://www.federalpay.org/employees
Ultimately I used the raw data set provided here: https://www.opm.gov/data/index.aspx (I downloaded the Fiscal Year 2015 data from August 28, 2016)
However they only provide the data in CSV files, so I imported them into a SQL Database and I've made that available for download here: https://sfhot.blob.core.windows.net/public/OpmFedScope-fy2015.zip
The ZIP Archive contains a SQL Server DB Backup file that can be restored on your local SQL Server instance.
The schema of the database, and an example query, can be found here: https://gist.github.com/aaronhoffman/ebc6468da8485d26aea4751034c83d5f
Hope this helps,
Aaron
Thursday, September 8, 2016
usaspending.gov download data archive as relational database
I recently attempted to create a data visualization based on the data available at usaspending.gov here: https://www.usaspending.gov/DownloadCenter/Pages/dataarchives.aspx
The data is available in .csv and .tsv formats, but I preferred a relational database to easily query and investigate the data.
I have requested that they make the data available in a relational format for ease of use, however I am not sure if or when that will happen.
In the meantime, I will provide my relational database to anyone who requests it. Just comment below, or email me.
-Aaron
The data is available in .csv and .tsv formats, but I preferred a relational database to easily query and investigate the data.
I have requested that they make the data available in a relational format for ease of use, however I am not sure if or when that will happen.
In the meantime, I will provide my relational database to anyone who requests it. Just comment below, or email me.
-Aaron
Sunday, April 6, 2014
News Aggregator Comparison - Article Source Count - Q1 2014
Below is a comparison of two popular political news aggregators, Drudge Report and Real Clear Politics.
The chart below compares news sites that are linked to by these aggregators and how the total link counts during Q1 2014 compare. Data was collected by crawling these sites continuously over the Q1 of 2014.
Some interesting data points:
1. In Q1 of 2014, Drudge never linked to nationalinterest.org, reason.com, www.chicagotribune.com, or www.huffingtonpost.com.
2. In Q1 of 2014, Real Clear Politics never linked to www.infowars.com, www.wnd.com, www.dailymail.co.uk, or www.breitbart.com.
3. Other large disparities in number of links include www.washingtonpost.com, dailycaller.com, www.weeklystandard.com, www.washingtontimes.com, and www.nytimes.com.
The chart below compares news sites that are linked to by these aggregators and how the total link counts during Q1 2014 compare. Data was collected by crawling these sites continuously over the Q1 of 2014.
Some interesting data points:
1. In Q1 of 2014, Drudge never linked to nationalinterest.org, reason.com, www.chicagotribune.com, or www.huffingtonpost.com.
2. In Q1 of 2014, Real Clear Politics never linked to www.infowars.com, www.wnd.com, www.dailymail.co.uk, or www.breitbart.com.
3. Other large disparities in number of links include www.washingtonpost.com, dailycaller.com, www.weeklystandard.com, www.washingtontimes.com, and www.nytimes.com.
Tuesday, July 16, 2013
United States General Election Dates
Below are the United States General Election dates (both presidential and mid-term) since 1968:
11/5/1968
11/3/1970
11/7/1972
11/5/1974
11/2/1976
11/7/1978
11/4/1980
11/2/1982
11/6/1984
11/4/1986
11/8/1988
11/6/1990
11/3/1992
11/8/1994
11/5/1996
11/3/1998
11/7/2000
11/5/2002
11/2/2004
11/7/2006
11/4/2008
11/2/2010
11/6/2012
11/5/1968
11/3/1970
11/7/1972
11/5/1974
11/2/1976
11/7/1978
11/4/1980
11/2/1982
11/6/1984
11/4/1986
11/8/1988
11/6/1990
11/3/1992
11/8/1994
11/5/1996
11/3/1998
11/7/2000
11/5/2002
11/2/2004
11/7/2006
11/4/2008
11/2/2010
11/6/2012
More Info:
Sunday, December 9, 2012
The Four Ways Money Can Be Spent, By Milton Friedman
In Chapter 4 of his book "Free to Choose" Milton Friedman describes the above matrix that depicts how the relationship between the benefactor and the beneficiary influences the exchange of goods. Below is the excerpt from the book explaining the matrix:
When you spend, you may spend your own money or someone else's; and you may spend for the benefit of yourself or someone else. Combining these two pairs of alternatives gives four possibilities summarized in the following simple table.
Category I in the table refers to your spending your own money on yourself. You shop in a supermarket, for example. You clearly have a strong incentive both to economize and to get as much value as you can for each dollar you do spend.
Category II refers to your spending your own money on someone else. You shop for Christmas or birthday presents. You have the same incentive to economize as in the first case but not the same incentive to get full value for your money, at least as judged by the tastes of the recipient. You will, of course, want to get something the recipient will like--provided that it also makes the right impression and does not take too much time and effort. (If, indeed, your main objective were to enable the recipient to get as much value as possible per dollar, you would give him cash, converting your Category II spending to Category I spending by him.)
Category III refers to your spending someone else’s money on yourself--lunching on an expense account, for instance. You have no strong incentive to keep down the cost of the lunch, but you do have a strong incentive to get your money’s worth.
Category IV refers to your spending someone else’s money on still another person. You are paying for someone else’s lunch out of an expense account. You have little incentive either to economize or to try to get your guest the lunch that he will value most highly. However, if you are having lunch with him, so that the lunch is a mixture of Category III and Category IV, you do have a strong incentive to satisfy your own tastes at the sacrifice of his, if necessary.
Sunday, November 11, 2012
State of Iowa Caucus and General Election Results For 2008 and 2012
Summary
The tables below contain the Iowa Caucus results for 2008 and 2012 and the General Election results for those same years.
Support for representatives of "libertarian" policies increased greatly. Support for Republican representatives also grew. Support for Democrat representatives shrank but maintained a majority of total votes cast.
Vote Count Compared to Total State Population
Iowa Republican Caucus 2008 and 2012
Iowa General Election 2008 and 2012
Analysis
Between 2008 and 2012, the total number of votes in these contests when compared to total state population remained relatively similar, however the change in percent of total vote (% Change) changed substantially for representatives of some groups.
For the Iowa Republican Caucus, between 2008 and 2012 Ron Paul had the greatest increase in number of votes (Count) and in change in number of votes (% Change). The only other candidate to compete in those same contests was Mitt Romney who received 216 fewer votes in 2012 than he did in 2008 (note: Romney did not actively campaign in Iowa for either Caucus).
For the Iowa General Election, between 2008 and 2012 the Republican Party had the greatest increase in number of votes (Count) and percent of total votes (% of Total), however the Libertarian Party had the greatest increase in change in number of votes (% Change).
For the Libertarian Party this trend was also seen nationally. The Libertarian Party received a total of 1,195,306 votes nationally, which was an increase of 128.24 % over 2008 and an increase of 200.88 % over 2004. Nationally, the change in number of votes (% Change) for the Libertarian Party has varied greatly since 1972 (the first General Election after its founding in 1971) however this election marked the longest sustained increase in percent of total votes (% of Total) the party has received.
Sources:
http://uselectionatlas.org/
http://en.wikipedia.org/wiki/United_States_presidential_election_in_Iowa,_2008
http://en.wikipedia.org/wiki/Iowa_Republican_caucuses,_2012
http://sos.iowa.gov/
http://en.wikipedia.org/wiki/Libertarian_Party_(United_States)
The tables below contain the Iowa Caucus results for 2008 and 2012 and the General Election results for those same years.
Support for representatives of "libertarian" policies increased greatly. Support for Republican representatives also grew. Support for Democrat representatives shrank but maintained a majority of total votes cast.
Vote Count Compared to Total State Population
Contest
|
Total Vote
Count 2008
|
Total Vote
Count 2012
|
Difference
2008 to 2012
|
Voter Turnout As % of
Estimated State Population
|
|||
% Change
|
2008
|
2012
|
% Diff
|
||||
Iowa Caucus
|
118,917
|
121,501
|
2,584
|
+ 2.17 %
|
3.96 %
|
3.97 %
|
+ 0.01 %
|
General Election
|
1,537,123
|
1,555,570
|
18,447
|
+ 1.20 %
|
51.19 %
|
50.80 %
|
- 0.40 %
|
Iowa Republican Caucus 2008 and 2012
Candidate
|
2008 Republican Caucus
|
2012 Republican Caucus
|
Difference
2008 to 2012
|
|||
Count
|
% of Total
|
Count
|
% of Total
|
% Change
|
||
Ron Paul
|
11,841
|
9.96 %
|
26,036
|
21.43 %
|
+ 14,195
|
+ 119.88
%
|
Mitt Romney
|
30,021
|
25.25 %
|
29,805
|
24.53 %
|
- 216
|
- 0.72
%
|
Rick Santorum
|
-
|
-
|
29,839
|
24.56 %
|
-
|
-
|
Mike Huckabee
|
40,954
|
34.44 %
|
-
|
-
|
-
|
-
|
Iowa General Election 2008 and 2012
Party
|
2008 General Election
|
2012 General Election
|
Difference
2008 to 2012
|
|||
Count
|
% of Total
|
Count
|
% of Total
|
% Change
|
||
Libertarian
|
4,590
|
0.30%
|
12,759
|
0.82%
|
+ 8,169
|
+
177.97 %
|
Republican
|
682,379
|
44.39%
|
722,754
|
46.38%
|
+ 40,375
|
+ 5.92
%
|
Write-In
|
6,737
|
0.44%
|
6,840
|
0.44%
|
+ 103
|
+ 1.53
%
|
Democrat
|
828,940
|
53.93%
|
807,394
|
51.81%
|
- 21,546
|
- 2.60
%
|
Analysis
Between 2008 and 2012, the total number of votes in these contests when compared to total state population remained relatively similar, however the change in percent of total vote (% Change) changed substantially for representatives of some groups.
For the Iowa Republican Caucus, between 2008 and 2012 Ron Paul had the greatest increase in number of votes (Count) and in change in number of votes (% Change). The only other candidate to compete in those same contests was Mitt Romney who received 216 fewer votes in 2012 than he did in 2008 (note: Romney did not actively campaign in Iowa for either Caucus).
For the Iowa General Election, between 2008 and 2012 the Republican Party had the greatest increase in number of votes (Count) and percent of total votes (% of Total), however the Libertarian Party had the greatest increase in change in number of votes (% Change).
For the Libertarian Party this trend was also seen nationally. The Libertarian Party received a total of 1,195,306 votes nationally, which was an increase of 128.24 % over 2008 and an increase of 200.88 % over 2004. Nationally, the change in number of votes (% Change) for the Libertarian Party has varied greatly since 1972 (the first General Election after its founding in 1971) however this election marked the longest sustained increase in percent of total votes (% of Total) the party has received.
Sources:
http://uselectionatlas.org/
http://en.wikipedia.org/wiki/United_States_presidential_election_in_Iowa,_2008
http://en.wikipedia.org/wiki/Iowa_Republican_caucuses,_2012
http://sos.iowa.gov/
http://en.wikipedia.org/wiki/Libertarian_Party_(United_States)
Subscribe to:
Posts (Atom)