Order of elements in datasource item XML is wrong

[Bartman0]

I wrote this issue report on the community forums, (post):

I am having trouble getting a data source published through the REST API.
The resulting error is 400011: Bad Request.

If I check the request against the 3.0 REST API XSD, it is valid:

According to XML Spy at least.

I changed the request_factory.py to do this:

class DatasourceRequest(object):
def _generate_xml(self, datasource_item, connection_credentials=None, connections=None):
xml_request = ET.Element('tsRequest')
datasource_element = ET.SubElement(xml_request, 'datasource')
datasource_element.attrib['name'] = datasource_item.name

if connection_credentials is not None and connections is not None:
raise RuntimeError('You cannot set both connections and connection_credentials')

if connection_credentials:
credentials_element = ET.SubElement(datasource_element, 'connectionCredentials')
credentials_element.attrib['name'] = connection_credentials.name
credentials_element.attrib['password'] = connection_credentials.password
credentials_element.attrib['embed'] = 'true' if connection_credentials.embed else 'false'

if connection_credentials.oauth:
credentials_element.attrib['oAuth'] = 'true'

project_element = ET.SubElement(datasource_element, 'project')
project_element.attrib['id'] = datasource_item.project_id

The fix is to have the project definition come after the credentials. Before this change, the request was not valid according to the XSD.

Overall, I am trying to achieve this:

• change database environment in the data source
• publish the data source with changed credentials
• I am using the document and service Python SDK's to do this

Script is attached.

datasource_publish.py.zip

[t8y8]

@irwando @RussTheAerialist the ordering of elements shouldn't matter -- this seems like a Server-side problem?

[Bartman0]

The ordering matters looking at the XSD. But the server should not enforce that, that's what you are saying?

I'm potentially seeing the same thing. When publishing to 2018.2 with server.version set to '3.0', I often get a BadRequest response from the server. If I catch that error, change the server version to '2.4' with the same DatasourceItem, then it goes through.

Other oddity that I haven't fully tested out: it appears that if I have multiple datasources to publish, all will fail with server.version set to 3.0. After adding the catch above on the first datasource and getting a successful publish, subsequent publish attempts, never get the BadRequest response. Things I know I haven't ruled out are publishing to different sites (using different auth tokens) and enough testing to be sure that it's not random. Pretty sure the first datasource always failed at server version set to '3.0'

[Bartman0]

Care to share code snippets? Much appreciated.

The relevant snippets:

import tableauserverclient as tsc

server = tsc.Server(server_address=XXXX)
server.version = '3.0'

with server.auth.sign_in(auth_req=tableau_creds): 
    # code to get project location and create DatasourceItem
    try:
        new_datasource = server.datasources.publish(datasource_item=datasource, file_path=filename, mode='Overwrite')
    except tsc.server.endpoint.exceptions.ServerResponseError:
        server.version = '2.4'
        new_datasource = server.datasources.publish(datasource_item=datasource, file_path=filename, mode='Overwrite')    

# remainder of method

FYI: went with 2.4 as that is what we had been using successfully in the past. I have not tried intermediate api versions.

edit: to add import of tableauserverclient in snippet

[jacalata]

Order does matter in xml for elements in a sequence. I can repro this locally with newer versions of Server as well.
https://stackoverflow.com/questions/4328867/does-xml-care-about-the-order-of-elements

[t8y8]

DatasourceItem responses are just elements as constructed by various serializers in Server, but also whatever language you happen to use (python here) -- I don't recommend that we make order part of the API contract, it could change if the underlying java library changes how it writes data out. If you use the JSON representation your even more likely to hit different element ordering because many hash maps/dicts aren't ordered.

If we use sequence in our XSD maybe that's incorrect and it should be something else that doesn't require ordering?

[bcantoni]

I don't believe there is anything to fix here, but will leave open to investigate that question about changing things in the XSD.

[jacalata]

The xsd sequence element generates this restriction on the Java class:
@XmlType(name = "customViewType", propOrder = {
"view",
"workbook",
"owner"
})

however this is only expected to affect the serialized output, not the acceptable input.

AFAICT, we could replace xs:sequence with xs:all in every case and the only effect should be removing this (not-supposed-to-exist) constraint on order of elements.