lda_api

LDA Reports API v1

About

This project is intended to parse through the Lobbying Disclosure Act (LDA) Reports. It currently supports all lobbying data endpoints, including filings, contributions, registrants, clients and lobbyists.

Code is in continuous development and is currently stable to parse through Reports stated above. However, there are still some issues and improvements planned in the near future, including adding authentication endpoints which are still not available in this code. See the API information

Use

Each endpoint is available as a struct with certain methods to directly ingest to MongoDB or retrieve raw structs to process and insert into other databases such as SQL. There are currently no plans to develop code for an SQL implementation, so we opted to leave a method to retrieve the raw structs.

Basic usage is as follows:

...

	api := api_v1.Config_api{Authorization: "your api key (can be left blank)", Mongo_URI: "mongodb://localhost:27017/", Database: "Lobbying_test", Collection: "v1"}

	error1 := api.Init_mongo()
	if error1 != nil {
		slog.Error("Handling error, initializing mongo")
		defer api.Close_mongo()
	}
	defer api.Close_mongo()
...

Users must first initialize the api struct, which will contain the API token provided by the LDA REST application; followed by their URI for their MongoDB service. We are also working on adding more features to this struct. Users may leave the Authorization blank as follows: Authorization: "", this will make the parsing of the information slower but not stop it. To increase the speed of pagination, users must request a Token key from the LDA website (this will eventually be made available as code for this library.)

After initializing the api struct, users may start querying the REST API. Methods Query_ingestion(config) and Retrieve(config) are part of the DataBaseProcess[T Result] interface, which can be used within another function to parse through multiple queries. Below are some examples, but we highly encourage users to see structs in the underlying code and values available for query in the REST API's documentation:

Contributions

...
    // multiple contributions
	query := api_v1.ListContributionReports{Contribution_honoree: "some individual", Filing_year: "2023", Contribution_contributor: "some company or entity"}

	err := query.Query_ingestion(api) // ingest directly to MongoDB
    if err != nil {
        // handle error
    }

    raw_structs, err := query.Retrieve(api) // retrieve raw structs, ideal for other types of databases users may be interested in
    if err != nil {
        // handle error
    }

    ...

    // individual contributions
    ind_query := api_v1.RetrieveContributionReport{Filing_uuid: "some unique ID"}
    
    err := ind_query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := ind_query.Retrieve(api) 
    if err != nil {
        // handle error
    }


...

Filings

	// multiple filings
	api.Collection = "filings" // change the collection to avoid multiple schemas

	query := api_v1.ListFilings{Foreign_entity_listed_indicator: "true", Filing_year: "2024"}

	err := query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

   // individual file
    ind_query := api_v1.RetrieveFiling{Filing_uuid: "some unique ID"}
    
    err := ind_query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := ind_query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

Registrants

    // Multiple registrants
	api.Collection = "Registrant"
	query := api_v1.ListRegistrants{Registrant_name: "Some company"}
    err := query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

    // individual registrant
    ind_query := api_v1.RetrieveRegistrant{Id: "some ID"}
    err := ind_query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := ind_query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

Clients

    // Multiple clients
    api.Collection = "Clients"
	query := api_v1.ListClients{Client_name: "Some name"}
    err := query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

    // single client
	ind_query := api_v1.RetrieveClient{Id: "some ID"}
    err := ind_query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := ind_query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

Lobbyists

    // Multiple lobbyists
	api.Collection = "Lobbyists"
	query := api_v1.ListLobbyists{Lobbyist_name: "some name"}
    err := query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

    // single lobbyist
	ind_query := api_v1.RetrieveLobbyist{Id: "some ID"}
    err := ind_query.Query_ingestion(api)
    if err != nil {
        // handle error
    }

    raw_structs, err := ind_query.Retrieve(api) 
    if err != nil {
        // handle error
    }

    ...

Future development

Development includes improving error handling, and incorporating POST requests for authentication endpoints.

Acknowledgements

Empower team authors: Samuel Rosado Zaidi and Daniel Mata