Questions and Answers from the Forward-Leading: Big Data and AI Summit

Jun 15, 2018   •   Lyndon Maydwell



I attended the Big Data and AI Leaders summit in Sydney at the end of April. Rather than simply plan to observe and network, I tried to make sure that I asked one question per speaker. I didn’t quite manage to achieve this, but got quite close.

While I was at the summit, I wrote down a list of questions for each speaker as I thought of them during their presentation, then asked what I thought was the best of the lot during post-talk question time. Below I’ll include the question and answers that were deliver, as well as a full list of candidate questions as an appendix. In addition to this I’ll include some methodology for asking questions at similar events in general.



Note - All these questions and answers were transcribed and partially recalled from memory. Therefore they can be considered more of a praraphrasing than a quote. If there are any mistakes of either phrasing or intent, then I apologise.

Day One - 27/04/2018

David at Commbank in Quantitative Services on Quantum Machine Learning

Question:

Will the high costs in quantum computing concentrating increase inequality between smaller and larger players in the space?

Answer:

Quantum computing is currently being made available already through cloud services, so the fear that the technology will be restricted to a few players is unwarranted.

Mark - Head of Data-Science at Canon on Data Visualisation

Question:

How do you prevent cross-contamination of data while gaining cross-silo insights into data?

Answer:

This is achieved through business-process design and data-governance.

Warwick Graco, Director of Data-Science at the ATO on Management Responsibilities as Opposed to Data Responsibilities

Question:

Should data-science managers be data-scientists themselves? Or are there general managers that can still perform the role well? Should a general level of understanding and competency of data-science be required of all managers who are operating in data-centric businesses and roles?

Answer:

Competency should be required. Analytics components should be included in management education to ensure this.

Jamie at CommBank on Data Governance from Inception to Implementation

Question:

The roadmap presented looks very linear and simple, is the pre-implementation process of data-governance actually this straight-forward? If not, what challenges are the most dangerous?

Answer:

Implementation and change management are absolutely the most challenging aspects of increasing an organisation’s data-governance maturity. Frameworks are easy.

Karthik Murugan, Head of Customer Analytics from Capgemini on Creating Data-Driven Customer Ecosystems

Question:

When should you consider data a liability rather than an asset? Is every touch-point to be optimised for data capture?

Answer:

You should optimise for good customer experience during touchpoints.

Tony at Sportsbet on Great Data Products

Question:

Is there a divide between providing good experiences and generating influence on customers? What are your thoughts on this in the context of the recent Cambridge Analytica events?

Answer:

We optimise for long-term revenue as an outcome of good customer-experience.

Day Two - 28/04/2018

Russel Nash - Customer Engineer, Google, on Big Data and Machine Learning with Google

Question:

Will there be saturation effects in the ML market as algorithms become more general? Commoditized saturated markets compete less and switch to a winner-takes-all / merger-acquisition phase - will this happen in ML like the ISP market? Is the ecosystem too rich for this to occur?

Answer:

No, Research will keep pushing the industry forward - plus open-source will empower individuals.

Rick Clark - ABS, on Emerging Data and Methods

Question:

Can incorrect or non-representative data be redacted? How?

Answer:

~ Bias truck location example, No social-media yet. ~ Broken ABN example, repair broken data through linkage.

Ash Nair - QBE

Question::

As insurance becomes more tailored, will expected return for individuals decrease? In the past technical hurdles prevented this. Can insurance make a shift from compensation to intervention with an increased depth of knowledge and prediction?

Answer:

Yes, IOT, etc. will allow us to jump before events.

Ferhad - Amaysim, on Building Advanced Analytics

Question:

Do you consider that UX is another stakeholder? Does this play a part?

Answer:

Nothing is currently done with regards to UX in analytics.

Peter McCallum - Head of Blockchain, Centrality, on Big Data Analytics meets Blockchain

Question:

How do you determine a legitimate application of blockchain? What criteria do you use? Do you take a qualitative or quantitative approach?

Answer:

Valid question - We ask - “Should multiple parties own the ledger?”

Ashkhen Zakharyan - Avia, on AI Music Composition

Question:

What is the latent vector for Aiva? Can you pick one that enforces specific relational rules, such as species conterpoint? What about cues for motifs in scores, etc.

Answer:

The ability to use cues for scoring is possible.

Luigi Barone - Daitum, on AI in Decision Automation

Question:

Are you using reinforcement learning techniques to reframe decision analytics problems? What about end-to-end training to minimise regret?

Answer:

Real world is reinforcement learning, but this isn’t seen a lot in operations.

Neil Fraser - James Cook University

Question:

Where do most enrolements come from? Are you seeing the cross-polination you would have anticipated/desired?

Answer:

Undergrad, Marketing, Comsci.

Eric Charran - Chief Architect, Microsoft, on The Ethics of AI

Question:

With regards to ethical considerations during research / testing, how do you determine when to hit the brakes early?

Answer:

We use the Wall Streat Journal test. However danger exists even if the WSJ test passes.





Appendix A: Asking Good Questions


As an aside, several people commented that they enjoyed my questions, so I thought I’d frame my thoughts on what makes a good question at an event such as this, but any event in general:

  • Make sure that the question relates strongly to the material.
    • As a guideline, if you’ve thought of the question ahead of time then it is probably not a good question.
    • Don’t tailor it to the individual, or the subject category alone.
  • Hide a second question inside your question
    • This doesn’t have to be answered, but it will provide an oportunity for the speaker to distinguish themselves by figuring it out.
  • Make you question challenging.
    • Not rude, but there should be a component of dilemma inside your question.
  • Keep your questions focused.
    • A trick is to write down several questions and pick the best one. Don’t try to merge the questions.
  • Don’t be too eager to ask the first question.
    • Wait for a few other questions to be asked first
    • Adapt your questions as a result if yours have already been answered.





Appendix B: Full List of Candidate Questions

Below follows a list of questions that were selected from for the ones actually asked to speakers.

Day One - 27/04/2018

David at Commbank in Quantitative Services on Quantum Machine Learning

Question One:

What in your opinion is the next Shore’s algorithm that will emerge? Something like quantum back-propagation?

Question Two:

How are these technologies that you are discussing placed on the Gartner hype-cycle that was referenced earlier?

Question Three:

Even though an enormous amount of parallel information can be stored and processed in a small number of q-bits, encoding of states for general computation requires significantly more bits than even the most ambitious estimates for q-bit availability in the near-future. Does this then in your opinion restrict the role of quantum computers purely to specialist activities?

Question Four:

Does the ‘no-cloning’ theorem require a combinatorial explosion in pre-quantum input marshalling for general purpose problems?

Question Five:

Are there dimensionality constraints on ediobatic optimisation? Could you consider some of these problems as a form of Tropical algebra?

Question Six:

Are quantum computers able to help overcome the data-bottlenecks that present themselves in dealing with massive data influx in areas such as particle-physics and astronomy?

Question Seven:

What is your specific role at CommBank?

Question Eight: (Asked and Answered)

Will the high costs in quantum computing concentrating increase inequality between smaller and larger players in the space?

Answer:

Quantum computing is currently being made available already through cloud services, so the fear that the technology will be restricted to a few players is unwarranted.

Question Nine:

Who will own the quantum computers of the future? Will data-privacy become an issue when you offload highly-computationally intensive tasks to quantum-clouds? How do you imagine that this will be addressed? Through some form of quantum-homomorphic encryption, or maybe via exploiting quantum encryption from the client to the cloud?

Tony Nolan at G3N1U5 on ‘Hacking the Universe’

Question One:

What are the tricks in creating multi-purpose datasets?

Question Two:

What are you trying to achieve with the plethora of ideas presented in these slides? Is there a focus besides novelty?

Question Three:

When should we not construct these large datasets? Do you worry about creating a Pandora’s box scenario?

Question Four:

How many members are there in the G3N1U5 group?

Question Five:

‘Chromathic’? Where are these terms coming from? Does this propose to somehow circumvent limitations of bit-level information transmission? How?

Question Six:

Are there precision issues during computation when performing calculation and data-transmission with these analog optical techniques? What about when marshalling data into and out of the analog context?

Mark - Head of Data-Science at Canon on Data Visualisation

Question One:

How do you rate the visualisation tools that you have been using and developed against the newer cloud services, such as AWS Quicksight?

Question Two:

What is the security model for this?

Question Three: (Asked and Answered)

How do you prevent cross-contamination of data while gaining cross-silo insights into data?

Answer:

This is achieved through business-process design and data-governance.

Question Four:

How do you achieve the rapid production deployment reference?

Question Five:

How can you ensure engineering standards and code quality are maintained as prototypes move from an experimental context into a production context?

Warwick Graco, Director of Data-Science at the ATO on Management Responsibilities as Opposed to Data Responsibilities

Question One:

Are we reinventing science in data-science?

Question Two:

What are the methods you can use to interrogate results and the people producing them to ensure that they meet your standards?

Question Three: (Asked and Answered)

Should data-science managers be data-scientists themselves? Or are there general managers that can still perform the role well? Should a general level of understanding and competency of data-science be required of all managers who are operating in data-centric businesses and roles?

Answer:

Competency should be required. Analytics components should be included in management education to ensure this.

Question Four:

Is there a mandate to communicate the subtleties of confidence and influence intuitively if you are tweaking cut-offs? How do you teach beysianism to customers?

Question Five:

Is the percentage of income-tax quote a red-herring? What about re-framing this as marginal-impact of income-tax on quality-of-life?

Question Six:

Can you deliver impartial statistics?

Question Seven:

What techniques can you use to identify applicable business outcomes where lower-accuracy models are still valuable?

Question Eight:

What is the psychometritian’s book that you referenced in your slides?

Question Nine:

Is there a meta-statistic on false diminishing-returns through ablation studies?

Jamie at CommBank on Data Governance from Inception to Implementation

Question One:

What is Data Governance? Are there different meanings to different people?

Question Two:

What is a motivation model? It addresses the “Why” by what is the model component?

Question Three: (Asked and Answered)

The roadmap presented looks very linear and simple, is the pre-implementation process of data-governance actually this straight-forward? If not, what challenges are the most dangerous?

Answer:

Implementation and change management are absolutely the most challenging aspects of increasing an organisation’s data-governance maturity. Frameworks are easy.

Question Four:

How is any of this validated? Are there situations where the plan doesn’t fit? Have you experienced any counter-examples?

Question Five:

How do you accumulate and select use-cases?

Question Six:

How do you construct a representative proof-of-concept for data-governance? This seems difficult.

Question Seven:

What is a communication plan in the data-governance context?

Karthik Murugan, Head of Customer Analytics from Capgemini on Creating Data-Driven Customer Ecosystems

Question One:

Is this plan linear?

Question Two:

How do you ensure neutrality is ‘customer centric’ and not have it fall into the trap of being a political playing-card?

Question Three:

Are there hidden ‘touch-points’? Who decides what the touch-points are?

Question Four (Asked and Answered):

When should you consider data a liability rather than an asset? Is every touch-point to be optimised for data capture?

Answer:

You should optimise for good customer experience during touchpoints.

Question Five:

Where do you draw the line in customer communications specialisation? How should this be determined?

Question Six:

Can the journey be parallelised?

Question Seven:

What is the thrust of this talk? Is it “Customer interaction and capture and levers”?

Tony at Sportsbet on Great Data Products

Question One:

Difficulty of implementation is listed as an advantage, but with the (inevitable?) trend towards easier and commoditized implementation, is this just a first-mover advantage?

Question Two:

Do you try to lock-in customers with methods besides good customer-experience?

Question Three:

What do you think about reverse-loyalty incentives?

Question Four: (Asked and Answered)

Is there a divide between providing good experiences and generating influence on customers? What are your thoughts on this in the context of the recent Cambridge Analytica events?

Answer:

We optimise for long-term revenue as an outcome of good customer-experience.

Question Five:

Can the economic pressure of competition possibly incentivise a trend towards nurturing rather than exploiting customers?

Question Six:

How is the expected margin set on odds, is this optimised for revenue? Do you provide transparency to your customers of simulated estimated probabilities? How does this fit into regulations?

Guzman, Khartic, Jamie - Panel

Discuss: New privacy considerations

Discuss: Maturity curve for models

Discuss: How has AI/ML changed your business?

Discuss: What is the opportunity cost for neglecting AI?

Discuss: What are the main challenges?

Discuss: How will personal life change?

Questions from the audience.

Question: How can we combat the perverse incentives of competitive advantage in relation to AI? Will the user be more influenced and more exploited? Is customer experience consideration enough to counter-act these forces?

Question:

Will AI help with data preparation?

Answer - Jamie:

Yes. In the same respects this is a customer experience question. We should address ethics as individuals and companies.

Question: Do users consider ethics? GMail example. GPS example. The burden on users.

Question:

Is the toll the master or is the tool the servant?

Answer - MC:

In a math scenario, this isn’t a problem.

Answer - Jamie:

AI literacy will reduce fear.

Day Two - 28/04/2018

Russel Nash - Customer Engineer, Google, on Big Data and Machine Learning with Google

Question One:

Where do the limits of Google’s “ecosystem nurturing” kick in? Does Google see AI as a public good? Is Google’s advantage maintained by being a first-mover?

Question Two:

What are your thoughts on NVIDIA pulling the rug out from under the ML community by enforcing enterprise hardware restrictions?

Question Three: (Asked and Answered)

Will there be saturation effects in the ML market as algorithms become more general? Commoditized saturated markets compete less and switch to a winner-takes-all / merger-acquisition phase - will this happen in ML like the ISP market? Is the ecosystem too rich for this to occur?

Answer:

No, Research will keep pushing the industry forward - plus open-source will empower individuals.

Question Four:

Is the notion of super-compilation applicable to ML in the sense that specialised contexts can be attached with specialised interpreters passed through a super-compilation phase?

Audience Question:

Is there ML Work in Higher-Education in Australia?

Answer:

Yes, but unfamiliar. Will put you in touch with some contacts.

Audience Question:

Are TPUs for sale?

Answer:

No. Cloud only.

Rick Clark - ABS, on Emerging Data and Methods

Question One:

Is there an analog of “evidence based policy” to “evidence based values”? What is the purpose of all of these points discussed and the ABS’s role?

Question Two:

Can the silo / wholistic problem be solved by reframing the problem?

Question Three:

Are these emergent properties universal, table and analysable? No?

Question Four:

How does this relate to the idea of reflexivity?

Question Five:

Fudged data-acquisition is prevented and remedied how?

Question Six:

How do you enable technology experimentation within large organisations like the ABS?

Question Seven: (Asked by another audience member)

How do you maintain data-sanity in a free-form graph context?

Answer:

Semantic-Web, OWL, SPQRQL

Question Eight:

Can upcoming motifs be predicted? Black-swan events made more tactile?

Question Nine: (Audience)

Who else will have access to this data?

Answer:

We would like to enable more access, but confidentiality is required. Sensitivity tags need to be implemented.

Question Ten: (Asked and Answered)

Can incorrect or non-representative data be redacted? How?

Answer:

~ Bias truck location example, No social-media yet. ~ Broken ABN example, repair broken data through linkage.

Question: (Audience)

How do you achieve isolation?

Answer:

A move away from Table Builder to a new technology

(unclear if I caught this answer correctly…)

Question: (Audience)

Does the ABS look at the impact of AI on jobs?

Answer:

Not directly, furthermore there is no policy on this.

Ashok Nair - Head of Data and Analytics, QBE

Question One: (Asked and Answered)

As insurance becomes more tailored, will expected return for individuals decrease? In the past technical hurdles prevented this. Can insurance make a shift from compensation to intervention with an increased depth of knowledge and prediction?

Answer:

Yes, IOT, etc. will allow us to jump before events.

Question Two:

Will insurance subsume other industries?

Question Three:

Is it too big a hurdle to tackle high-level abstract ideas?

Ferhad - Amaysim, on Building Advanced Analytics

Question One:

Did Amaysim scale faster than expected? Were the technical compromises that were made early on painful to deal with later? How do you remain pragmatic, but prepare for this eventuality?

Question Two:

Are the steps on the AI journey agreed on or proven? Could there be more steps, or other paths?

Question Three: (Asked and Answered)

Do you consider that UX is another stakeholder? Does this play a part?

Answer:

Nothing is currently done with regards to UX in analytics.

Question Four:

What is D&A?

Question Five:

Outrix? Kinesis, Athena, Brytlyt.

Question Six: (Asked by Audience)

Did you experience data incoherence in your lake? How did you combat semantic confusion without a schema?

Answer:

AWS Glue.

Question Seven:

Do you monitor for weirdness in your data-lake and how do you respond to contamination?

Question Eight:

What are some concrete insights that the pipeline has delivered that you are now leveraging?

Peter McCallum - Head of Blockchain, Centrality, on Big Data Analytics meets Blockchain

Question One:

Is centrality a sinister name in the blockchain context?

Question Two: (Asked and Answered)

How do you determine a legitimate application of blockchain? What criteria do you use? Do you take a qualitative or quantitative approach?

Answer:

Valid question - We ask - “Should multiple parties own the ledger?”

Question Three:

Are you worried about being too dependent on AWS? What’s your escape strategy?

Question Four:

Have you been bitten by NLP subtleties? How do you quantify regret?

Question Five:

How do you weigh sentiment by source an how do you quantify and integrate source bias?

Question Six: (Asked by Audience)

How would you project a chain for analytics? Time series DB?

Answer:

This seems to be yes - RDBMS, Graph DB - Example: Blockchain on Big Query

Question Seven:

Shouldn’t value be determined by underlying value rather than speculation?

Question Eight: (Audience)

How do you handle fork-monitoring?

Ashkhen Zakharyan - Avia, on AI Music Composition

Question One:

Are you familiar with EMI?

Question Two:

How are you addressing the issues with shallow / local composition phenomena in music?

Question Three:

Is the fundamental problem of creative AI establishing a creative loss function? Is direct semi-supervised creativity ill-conceived?

Question Four:

Personalisation depends on deep-insight into the individual. How do you quantify an individual in a useful way in this context?

Question Five: (Asked after the Session)

What is the latent vector for Aiva? Can you pick one that enforces specific relational rules, such as species conterpoint? What about cues for motifs in scores, etc.

Answer:

The ability to use cues for scoring is possible.

Question Six:

Is instrumentation indicated on the produced score? What about the arrangement?

Luigi Barone - Daitum, on AI in Decision Automation

  • Reference to Case-Study: Fleet Management - Logistics
  • Reference to Case-Study: Production Scheduling - Mining
  • Reference to Case-Study: Inventory Management - Energy

Question One: (Asked and Answered)

Are you using reinforcement learning techniques to reframe decision analytics problems? What about end-to-end training to minimise regret?

Answer:

Real world is reinforcement learning, but this isn’t seen a lot in operations.

Pitfalls:

  • Problem Choice
  • Data Integrity
  • Tech Hype
  • Outsourcing Business Knowledge
  • Understanding Disruption to Culture

Question Two:

What are the heuristics for ‘pattern translation’? Smart people? Anything else?

Question Three:

How do you quantify robustness?

Neil Fraser - James Cook University, on Applied Data-Science in Business

Question One:

In the context of “certified by SAS acadamy of data-sciences”, What does such a certification convey? Who has failed the certification?

Question Two:

Will Data-Science growth come from expansion of existing practices or new technology and techniques?

Question Three:

You would like to see the industry be more certified. What should happen to transform the industry as it currently stands?

Question Four:

How do your industry partnerships integrate prepresentatively? Is this a concern?

Question Five:

What are your specialties beyond visualisation? Agile research? ML UX? Other?

Question Six: (Asked and Answered)

Where do most enrolements come from? Are you seeing the cross-polination you would have anticipated/desired?

Answer:

Undergrad, Marketing, Comsci.

Question Seven:

What are your thoughts about universities moving in to the consulting space? Is this a conflict of interest?

Question Eight:

Can you get in touch for cap-stone project staff?

Question Nine:

Do these businesses have the competency to maintain and operate the models you have deployed? Do you recommend that they retain staff?

Eric Charran - Chief Architect, Microsoft, on The Ethics of AI

Question One:

How do you educate business about how to accept the possibility of failure in ML research experiments?

Question Two:

Should new business models be vetted with the old methodologies?

Question Three:

Is there a new role in business education for data-science / machine-learning?

Question Four: (Asked and Answered):

With regards to ethical considerations during research / testing, how do you determine when to hit the brakes early?

Answer:

We use the Wall Streat Journal test. However danger exists even if the WSJ test passes.

Question Five:

Do we really expect the industry to self-regulate? Imilar to Yesterday’s question - What about smaller companies? Is negotiating the ethical landscape an undue burden and unfair and unrealistic for them to achive versus big corporations such as Microsoft?

Question Six:

Are you serious with regards to people requiring quarterly reskilling in the future?




Hopefully this provides some insight into how I selected the questions that I asked the speakers at the summit. If you’d like to know more about how Silverpond participating in the big-data and AI space, then please contact us.






Level 2
382 Little Collins Street
Melbourne VIC, 3000

Enter from McKillop Street

Contact us

(03) 9008 5922

Get the latest on machine learning and data science in your inbox each month